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This is а solution manual for the textbook Linear Algebra and Its Applications, 2nd Edition, by Peter 
Lax (John Wiley & Sons, 2007). This version omits the following problems: Exercise 2, 9 of Chapter 8; 
Exercise 3 of Appendix 3; Exercise problems of Appendix 4, 5, 8 and 11. 


1 Fundamentals 


į; 
Proof. Suppose 0 and 0’ are two zeros of vector addition, then by the definition of zero and commutativity, 
we have 0/ = 0'+0=0+0'=0. o 
2. 


Proof. For any т = (21,::: 24) € К", we have 


т+0 = (ту, tn) +(0,--- ,0) = (£1 +0,--- 2 +0) = (1,24) = T. 


So 0 = (0,--- ,0) is zero element of classical vector addition. a 
3. 
Proof. The isomorphism Т can be defined as T((a,,--- ,a,)) = a, - agz +--+ + ат"). o 
4. 


Proof. Suppose S = {81,--+ , Sn}. The isomorphism T can be defined as T(f) = (f(s1),--- , f(8n)) Vf € 
KS. o 


5. 
Proof. For any р(х) = a1 + аат +- -- + алт"), we define 
Т(р) = p(z), 


where p on the left side of the equation is regarded as a polynomial over R while p(x) on the right side of 
the equation is regarded as a function defined on S = {s1,--- ,8n}. To prove T is an isomorphism, it suffices 
to prove T is one-to-one. This is seen through the observation that 


-—r 
pom pocos) fa] po) 
15 $5) зу a2| _ | p(s2) 
l ва Sa 7 R] Lay p(ss), 
and the Vandermonde matrix E 
1 5 s gat 
1 s 8 aot 
1 s 82 at 
is invertible for distinct зу, s2, ---, Sn- a 


6. 


Proof. For any y, y! € Y, 2,2 € Z and k € K, we have (by commutativity and associative law) 


(y*2*G te) = (:+)+(у+)=+(у+(у +2) =2t (0+0) +7) = + (# + (uty) 
= (z+z)+(y+y)=(y+y)+(z+27)EY +Z, 
and 
k(y +2) = ky +kz E€ Y +Z. 
So Y + Z is a linear subspace of X if Y and Z are. п 


T. 


Proof. For any 23,22 € Y N Z, since Y and Z are linear subspaces of X, тү + т: € Y and zi +2» € Z. 
Therefore, тү +22 € YNZ. For any k € K and т € Y NZ, since Y and Z are linear subspaces of X, kz € Y 


and kx € Z. Therefore, kr € Y П Z. Combined, we conclude Y П Z is a linear subspace of X. п 
8. 
Proof. By definition of zero vector, 0+0 = 0 € {0}. For any k € К, k0 = k(0+0) = k0--k0. So k0 = 0 є {0}. 
Combined, we can conclude {0} is a linear subspace of X. o 
9. 


Proof. Define Y = {kızı +-+- + Куту: k,--,k; € K}. Then clearly тү = 1z; + 025 +--+- +00; € Y. 
Similarly, we can show z2, ++: ,; € Y. Since for any ki, +++ kj, kj, TA € K, 


(kızı + +++ + Куту) + (kizi +++ jm; 


= (kı + hy )ai +--+ (kj + ki )ay € Y 
and for any kj, , kj, k € К, 
k(kizı +--+ + Куту) = (eki)m + +--+ (ЕК); EY, 


we can conclude Y is a linear subspace of X containing ту,--- ‚ту. Finally, if Z is any linear subspace of 


X containing z,::- , 7j, it is clear that Y C Z as Z must be closed under scalar multiplication and vector 
addition. Combined, we have proven Y is the smallest linear subspace of X containing 21,::· , xj. о 
10. 


Proof. We prove by contradiction. Without loss of generality, assume ту = 0. Then 1z; + 0x2 +--+ +02; = 
0. This shows z1,:::,7; are linearly dependent, a contradiction. So ту # 0. We can similarly prove 


72,:7,0j #0. o 
п. 
Proof. Suppose Y; has а basis yj,--- ‚у. Then it suffices to prove y]. --- yl, Ps: ym, form a basis 


of X. By definition of direct sum, these vectors span X, so we only need to show they are linearly independent. 
In fact, if not, then 0 has two distinct representations: 0 = 0 + --- +0 and 0 = У" (9101 +- + ah Yh) 


for some a}, +: ‚а, ,аү',--- , a7. , where not all аў are zero. This is contradictory with the definition 
of direct sum. So we must have linear independence, which imply y}, ++- yz, Ps ym, form a basis 
of X. Consequently, dimX = У dimY;. o 
12. 

Proof. Fix a basis T1,- -- ‚т, of X, any element т € X can be uniquely represented as У), o;()7; for some 
aj(x) € K, i = 1,--- ,n. We define the isomorphism as т ++ (a;(x),--- ,an(z)). Clearly this isomorphism 
depends on the basis and by varying the choice of basis, we can have different isomorphisms. o 
13. 


Proof. For any 21,22 € X, if z, = 29, i.e. тү — r € Y, then z2 — ту = —(ri — то) € Y, ie. z2 = z,. This 
is symmetry. For any т € X, z—z = 0 € Y. So т = т. This is reflexivity. Finally, if тү = £2, тә = 13, then 
тү — T3 = (£1 — T2) + (£2 — тз) € Y, ie. тү = ту. This is transitivity. a 


14. 
Proof. For any 21,22 € X, we can find y € {ту} п (22) if and only if zı — y € Y and z2 — y € Y. Then 

T1 — T2 = (T1 — y) - (12 — y) € Y. 
So (21) N {x2} #0 if and only if {z1} = {x2}. o 
15. 
Proof. If {x} = {2} and {y} = {y'}, then z—2',y —y' € Y. So (z+y)- (x y) = (z-2) (у-у) EY. 
This shows {x + y) = (z' + y'). Also, for any k € K, kr — kz' = k(x —a') € Y. So k{x} = {Кт} = {к} = 
k{x'}. o 
16. 
Proof. By theory of polynomials, we have 


Y= fo Пе — ti) : q(t) is a polynomial of degree < n — i} Я 


isl 


Then it's easy to see dim Y = n — j and dim X/Y = dim X — dimY = j. п 
17. 
Proof. By Theorem 6, dim X/Y = dim X — ішу = 0, which implies X/Y = {{0}}. So X = У. o 
18. 


Proof. Define Y; = {(2,0) : 2 € X1,0 € Хә} and Yz = {(0, 1) : 0 € X;,z € X2}. Then Y; and Yz are linear 
subspaces of Ху Ф X». It is easy to see Y; is isomorphic to Xj, Yz is isomorphic to X2, and Ү, NY2 = {(0,0)}. 
So by Theorem 7, dim X; Ф Хз = dimY; +dimY2—dim(Y, ПУ) = dimX; -dimX5—0 = dimX,+dimX2. O 


19. 
Proof. By Exercise 18 and Theorem 6, dim(Y @X/Y) = dimY --dim(X/Y) = іму +dimX—dimY = dimX. 
Since linear spaces of same finite dimension are isomorphic (by one-to-one mapping between their bases), 
Y 6 X/Y is isomorphic to X. o 
20. 


Proof. (a) is not since {x : zı > 0} is not closed under the scalar multiplication by —1. (b) is. (c) is not 
since ту + тә + 1 = 0 and zi +25 +1 = 0 imply (x; + z1) + (2 + 25) + 1 = —1. (d) is. (e) is not since zı 
being an integer does not guarantee rz; is an integer for any r Є R. g 


21. 
Proof. See the textbook's solution, page 279. a 


2 Duality 
1. 


Proof. Suppose e1, ---, €, is a basis of X and suppose ту = У)", о;е;. If the underlying field is R, we 
define a linear function І by setting [(е:) = а: (i = 1,--- n) and extending its definition to X via linear 
combination. If the underlying field is C, we define / similarly by setting l(e;) = at, where а? is the complex 
conjugate of a; (i = 1,--- ,n). In either case, we have I(x1) = |z1]? 3 0, where | - | is the Euclidean norm 
of R" or С". 

To generalize the above result to a general linear vector space X over a field К, we clearly need some 
notion of norm. This is exactly the starting point of Hahn-Banach Theorem, which claims a similar result 
for general linear vector spaces, not necessarily finite-dimensional (see Lax [6]). So this exercise problem 
needs extra conditions if we need to go beyond K = R or K — C. a 


2. 


Proof. For апу lı and lz € Y+, we have (lı +12)(y) = lh (y)+l2(y) = 0+0 = 0 for any y € Y. Sol; +l2 € Y+. 
For any k € К, (kl)(y) = k(l(y)) = k0 = 0 for any y € Y. So kl € Y^. Combined, we conclude Y^ is a 
subspace of X". п 


3. 


Proof. Since S C Y, Y^ C SŁ. For “>”, let z1,--- ,z,, be a maximal linearly independent subset of S. 
Then S = span(zi,:-- ,2,,) and Y = {У от; : 03,::: ,@m € K} by Exercise 9 of Chapter 1. By the 
definition of annihilator, for any | € S+ and y = 7", aia; € Y, we have 


Uy) = >> ails) 
i=l 


Sol € Y^. By the arbitrariness of І, St C Y+. Combined, we have S+ = Y. o 
4. 


Proof. Suppose three linearly independent polynomials pı, p; and рз are applied to formula (9). Then mı, 
mz and тз must satisfy the linear equations 


ра) Pills) pit) fra] [amt 
pa(h) р›(%в) p2(ts)| | тә | = | f_, p2(t)dt 
pa(&) Pa(t2) pa(t3)] [ma "руй 


We take pi(£) = 1, po(t) = t and p;(t) = 2. The above equation becomes 


тү 1117772] fo -2 г з 
m| =|-a 0 a 0-2] 0 -4 0|212 
ma аё 0 а? 3 0 x m E 3 


Then it's easy to see that for a > V/1/3, all three weights are positive. 
To show formula (9) holds for all polynomials of degree < 6 when a = 4/375, we note for any odd n € №, 


1 
/ z"dr = 0, тур(—а) + map(a) = 0 since m, = то and p(—x) = —p(z), and map(0) = 0. 
—1 


So (9) holds for any 2" of odd degree n. In particular, for p(x) = z? and p(x) = 2°. For p(x) = z*, we have 


* 2 2 

[2842 =. тч) + тарбы) + тарбы) = mia! = 2a. 
a 

So formula (9) holds for p(x) = z4 when а = \/3/5. Combined, we conclude for a = \/3/5, (9) holds for all 

polynomials of degree « 6. o 

Remark 1. In this exercise problem and Exercise 5 below, “Theorem 6” should be corrected to “Theorem 
qu. 

5. 

Proof. We take pi(f) = 1, pa(t) = t, pa(t) = #2, and p, (f) = t. Then mi, ma, тз, and m4 solve the following 
equation: 


1 1 1 1] jm 2 
-a -b b a} {mz} _ |] 0 
a b» P a?! |m3} {2/3 
-a 58 B a| |m 0 
Then 
m ЛГ? 
m| _ b a 0 
m| ~ ? а? 2/3 
ma з P а} 0 
b? 1 1 
2а5—3аї 07—008 —2а1+24Ь7 2 
a 1 1 
А Заль ал001 2475-37 0 
а 1 1 2/3 
c 1.78 iudi 0 
=2a5+2ab? — 247—259 2a Зар? 


So the weights are positive if and only if one of the following two mutually exclusive cases hold 


1) 2 > 1,а2 < 02, а? > 1; 
<i 


2) b? < ġa? >b, a? < 

6. 

Proof. (from the textbook’s solution) (a) Suppose there is a linear relation 
al (p) + bl;(p) + сіз(р) = 0. 

0, pı (&) # 0; so we get from the above relation that 


Set p = p(a) = (т — €2)(a — £3). Then p(£2) = p(£3) 
а = 0. Similarly b = 0, c = 0. 
(b) Since dim Р» = 3, also dim Р; = 3. Since l, l2, Їз are linearly independent, the span Pj. 
(cl) We define by setting 
1 #ј=1 
0, iff 41 


h(e;) 


and extending lı to V by linear combination, ie. (3%, aje; 


Уу гаје) = ол. lə, +++, In can be 
constructed similarly. If there exist a1, ---, a, such that а + 


++ anln = 0, we have 
0 —ajl(ej) +---anln(ej) = aj, j — 1, n. 


So lı, +-+- , In are linearly independent. Since dim V’ = dim V = n, (l,--- ,L,) is a basis of V". 
(c2) We define 


(ж — z2)(r — 3) 
(1 = тә)(ж1 = 23)" 


Pise (z — т1)(т — тз) 


ыст ыс: 


Pilz) = 7 Gy 21) y= za) 


т. 
Proof. (from the textbook’s solution) I(x) has to be zero for т = (1,0, —1, 2) and x = (2,3, 1, 1). These yield 
two equations for c, ---, c4: 

ср — єз + 2с = 0, 20 + 305 + сз + c4 = 0. 


We express c; and c» in terms of єз and c4. From the first equation, cı = сз — 2c4. Setting this into the 
second equation gives c) = —c3 + c4. n 


3 Linear Mappings 
1. (a) 


Proof. For any y,y/ € T(X), there exist z,z' € X such that T(x) = y and T(z’) = y. Soy y = 
T(x) + T(z") = T(r + 2’) € T(X). For any k € К, ky = КТ(т) = T(kr) € T(X). Combined, we conclude 
T(X) is a linear subspace of U. o 


(b) 
Proof. Suppose V is a linear subspace of U. For any x, a’ € Т- (V), there exist у,у € V such that T(z) = y 


and T(x’) = у. Since T(x + 2’) = T(z) + T(2') = y +y' € V, z-- z' € T-!(V). For any k € K, since 
T(kx) = kT(z) = ky € V, kz € T- (V). Combined, we conclude T! (V) is a linear subspace of X. [и] 


2. 


Proof. (from the textbook’s solution) Suppose we drop the ith equation; if the remaining equations do not 
determine т uniquely, there is an x that is mapped into a vector whose components except the ith are zero. 
If this were true for all i = 1,--- ,m, the range of the mapping 2 — и would be m-dimensional; but according 
to Theorem 2, the dimension of the range is < n < m. Therefore опе of the equations may be dropped 
without losing uniqueness; by induction m — n of the equations may be omitted. 

Alternative solution: Uniqueness of the solution z implies the column vectors of the matrix T — (t;;) are 
linearly independent. Since the column rank of a matrix equals its row rank (see Chapter 4), it is possible 
to select a subset of n of these equations which uniquely determine the solution. 


Remark 2. The textbook’s solution is a proof that the column rank of a matrix equals its row rank. 

3. (i) 

Proof. SoT(ax+by) = S(T(az + by)) = S(aT(z) + bT(y)) = aS(T(z)) + bS(T(y)) = aS o T(z) +bS o T(y). 

So S о Т is also a linear mapping. a 
(ii) 

Proof. (В + S) e T(z) = (В + $)(T(z)) = R(T(z)) + S(T(z)) = (Ro T + S o T)(z) and So (T + Р) 

ST + PG) = S(T(@) + P(z) = S(T(2)) + 8(Р(т)) = (8°Т+ 8 о PG). 


4. (a) 


Proof. Linearity of S and T is easy to see. For non-commutativity, consider the polynomial s. Then 
TS(s) = T(2) 22s # s = S(1) = ST(s). So ST #TS. o 


(b) 


Proof. For any x = (11,12,13) € X, S(x) = (11,13,12) and T(x) = (za,2, —71). So it's easy to see 
S and Т are linear. For non-commutativity, note ST(z) = S(£3, £2, —21) = (£3, —71, —72) and TS(z) = 
T (a1, 23, —72) = (—тә,тз,—ту). So ST # TS in general. п 


Remark 3. Note the problem does not specify the direction of the rotation, so it is also possible that S(x) = 
(21, 13,1) and T(x) = (—x3,22,21). There are total of four choices of (5,Т). But the corresponding 
proofs are similar to the one presented here. 


5. 
Proof. TT-\(«) = T(T-}(£)) = т by definition. So TT-! = id. a 
6. (i) 


Proof. Suppose T : X — U is invertible. Then for any у, € U, there exist а unique a € X and a unique 
a’ € X such that T(x) = y and T(z’) = у. So T(x + a") = T(x) + T(x’) = y + y' and by the injectivity of 
T, T-My - y) = zz! = T~! (y) + T- (y). For any k € К, since T(kx) = kT (x) = ky, injectivity of 2 
implies T^! (ky) = kr = КТ! (у). Combined, we conclude T is linear. 

(ii) 
Proof. Suppose T : X — U and S : U — V. First, by the definition of multiplication, ST is a linear map. 
Second, if z € X is such that ST(x) = 0 € V, the injectivity of S implies T(x) = 0 € U and the injectivity 
of T further implies т = 0 € X. So, ST is one-to-one. For any z € V, there exists y € U such that S(y) = 
Also, we can find x € X such that T(x) = y. So ST(x) = S(y) = z. This shows ST is onto. Combined, we 
conclude ST is invertible. 

By associativity, we have (ST)(T-1S-!) = ((ST)T-)S-! = (S(TT-!))S-! = SS-! = idy. Replace 
S with T-! and T with S-!, we also have (T-1S-!)(ST) = idx. Therefore, we can conclude (ST)-! = 
qrig- 
7. (i) 
Proof. Suppose Т: X — U and S : U — V are linear maps. Then for any given ГЄ V’, ((ST)'l,z) = 


(1,572) = (S'l, Tz) = (T'S'l,z), Vr € X. Therefore, (ST)'l = T'S'l. Let l run through every element of 
V’, we conclude (STY = T'S’. п 


(ii) 
Proof. Suppose Т and R are both linear maps from X to U. For any given l € U’, we have ((T + R)l,z) = 


(L, (T + R)z) = (l, Tx + Rz) = (l, Tx) + (l, Rz) = (T'l,z) + (Rl, £) = ((Т' + R),,z), Ут € X. Therefore 
(T +R) = (Т' + R’)L. Let 1 run through every element of V’, we conclude (T + R)’ = T' + R’. a 
(iii) 
Proof. Suppose T is an isomorphism from X to U, then T-! is a well-defined linear map. We first show 
T' is an isomorphism from U’ to X". Indeed, if 1 € U' is such that T’l = 0, then for any z € X, 0 = 
(T'lz) = (l, Tz). As т varies and goes through every element of X, Tz goes through every element of 
U. By considering the identification of U with U”, we conclude | = 0. So Т” is one-to-one. For any given 
m € X', define | = mT-!, then | € U’. For any x € X, we have (т, х) = (m, T (Tz)) = (I, Tz) = (T'l,z). 
Since т is arbitrary, m = T'I and T" is therefore onto. Combined, we conclude T" is an isomorphism from 
U' to X' and (T")-? is hence well-defined. 
By part (i), (T-2)'T' = (ТТ-1) = (idu)! = idy and Т(Т-1) = (T-!T) = (idx)! = idx. This shows 
(тту = (тус. п 
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8. 


Proof. Suppose € : X — X" and т: U — U” are the isomorphisms defined in Chapter 2, formula (5), which 
identify X with X" and U with U”, respectively. Then for any z € X and l € U’, we have 


(T"£,,D) = (€, T'I) = (T'L £) = (L, T£) = (тт, 1). 


Since | is arbitrary, we must have T”, = nrz, Vr € X. Hence, Т” оё = ro T, which is the precise 
interpretation of T" — T. п 


9. 


Proof. If Bx = 0, by applying A to both sides of the equation and АВ = I, we conclude z = 0. So B is 
injective. By Corollary B of Theorem 2, B is surjective. Therefore the inverse of B, denoted by B-!, always 
exists, and A = А(ВВ-!) = (AB)B-! = IB-! = B-!, which implies BA = I. o 


Remark 4. For a general algebraic structure, e.g. a ring with unit, it’s not always the case that an element’s 
right inverse equals to its left inverse. In the proof above, we used the fact that for finite dimensional linear 
vector space, a linear mapping is injective if and only if it’s surjective. 


10. 

Proof. Suppose K = Ms. Then К(М-!)ѕ = SMS-!SM-!S-! = I. By Exercise 9, K is also invertible 
and K^! = (M^7!)s. o 
п. 

Proof. Suppose A is invertible, we have АВ = АВ(АА-!) = A(BA)A-!. So AB and ВА are similar. The 
case of B being invertible can be proved similarly. о 
12. 

Proof. For any a, 8 € К and x = (z1,:--,24), y = (yi, 7: ‚уп ), we have 


Plax + By) = P((aa1+ By,- ‚ат + Byn)) 

(0,0, отз + Bys, , an + Byn) 

= (0,0, ar3,--+ ,a2n) + (0.0, ys, , Byn) 
(0,0, 23,+++ 25) + 8(0,0 9a. , Yn) 

= aP(r) +8Р(у). 


[i 


This shows P is a linear map. Furthermore, we have 


P?(z) = P((0,0,23,--- ,£n)) = (0,0, £3, --- 24) = Р(т). 


So P is a projection. [m] 
13. 

Proof. For any a, 8 € К and f,g € C[-1, 1], we have 

Р(аў+йд)(а) = Slaf+80)(2)+(af+80)(—2)] = S UG) Cx) ls) Ca] = aPU)G) SP). 


This shows P is a linear map. Furthermore, we have 


(PP) 


у) =P (A 


[e x f(x), f(-2) + f(x) 
2 2 


0а) fC2) = (РЈ). 


So Р is a projection. o 
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14. (a) 


Proof. Since dim Rz = 1, it suffices to prove the following claim: if T is a linear map on a 1-dimensional 
linear vector space X, there exists a unique number c such that T(x) = cr, Vr € X. We assume the 
underlying filed К is either R or C. We further assume S : X — К is an isomorphism. Then S o T o $7! is 
a linear map on К. Define c = So T o S^! (1), we have 


SoToS (k) = SoToS (k-1) = К.с, Vk € К. 


So T o S-!(k) = S-!(c- К) = cS-! (k), Vk € К. This shows T is a scalar multiplication. п 
(b) 

Proof. If c #1, it’s easy to verify I + т1-Т is the inverse of I — T. o 

15. 


Proof. Because Ёвт С Rs, dim(Rsr) < dim Rs. Also, we note Nsr/Nr is isomorphic to Ns N Rr, with 
the isomorphism defined by ó((z)) = Тт, where {т} := z + Nr. It’s easy to see ó is well-defined, is linear, 
and is both injective and surjective. So by Theorem 6 of Chapter 1, 


dim Nsr = dim Np + dim Nsr/Nr = dim Np + dim(Ns N Rr) < dim Np + dim Ns. 


4 Matrices 
L 


Proof. It looks the phrasement of the exercise has a problem: when m # n, AD or DA may not be well- 
ті 


defined. So we will assume т = п in the below. We can write А in the row form та. . Then DA can be 


Tm. 
written as 
d. 0 - O] fn diri 
DA = | 9. dae 9. | [rsen] dara 
Os: dj [ra алт, 
We can also write А in the column form [c1,¢2,--- ,Cn], then AD can be written as 
d o0 6 0 
AD = |с, оз, en] i @ i 2 = [da d202,: +- due] 
0 0 ++ d, 
м! 
2. 


Proof. Proofs in most textbooks are lengthy and complicated. For a clear, although still lengthy, proof, see 
Qiu [10], Theorem 3.5.3, page 112. п 


3. 
Proof. The calculation is a bit messy. We refer the reader to Qiu [10], Theorem 4.6.1, page 190. o 
4. 
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1 b 1 2 1 1 

Proof. шл=[, ol maB- |, М. Then 4B = oye Ba = [, ME п 
5. 
Proof. 

12 3 -1][1 1-14+2-243-(-2)+(-1)-1 -2 

254 -3 2| |2-145-24+4-(-2)+(-3)-1] |1 

234 1||-2| | 2-14+3-244-(-2)41-1 | | 1 

142 -2]|1 1-144-242-(-2)+(-2)-1 3 

o 

6. 
Proof. We choose ш = из = из = 1 and ug = 2. Then тз = —5r4 — из — u2 + Зи = —5r4 + 1, 


тз = 714 + u4 — Зи = 7z4 — 1, and ту = uj — T2 — 213 — 34 = 1 — (74 — 1) — 2(-524 + 1) – 374, = 0. O 


T. 


Proof. 
11283 
1231 
2-1/0 1253 
3462 
= [1.1-2.1-1.241-3,1.1-2-2—1-141-4,1.2-2-3—- 1-2 1:6,1.3-2.1—- 1-34 1:2] 
= 0. 
[m] 
8. 


Proof. Suppose a row vector т = (£1, £2, £3, £4) satisfies rM = 0. Then we can proceed according to 
Gaussian elimination 


T1 + T2 + 223 +324 = 0 


T2 — 734 та = 0 
тз +222 4-23 +44 = 0 % м, ы у тїз ==жа = 0 
Элу + Bay +225 + 6x4 =0 пе Е ШШ! -513 — 524 = 0. 
321 + 22 + Заз +224 = 0 2 4 r 
So we have тү = T4, тз = —2r4, and 3 = —24, ie. т = z4(1, -2, —1, 1), a multiple of l in Exercise 7. 
ui 
Equation (22) has a solution if and only if u = |”? | is in Ам. By Theorem 5 of Chapter 3, this is equivalent 
3 
ua 
to уи = 0, Vy € Nyy (elements of Ny are seen as row vectors). We have proved y is a multiple of l. Hence 
condition (23), which is just lu = 0, is sufficient for the solvability of the system (22). o 
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5 Determinant and Trace 


1. (i) 
Proof. By formula (5), we have |Pzi,--- та) = 1б: — ту)| = [Tags Gs — 25) = Porn. ,®һ))|. 
„+++ ота). Combined, we conclude |o(p)|=1. O 


By formula (6), we have |P(p(1,--- ,2tn)| = |o(p)||P(r 
(ii) 
Proof. By definition, we have 
Р(р о pa(zi, --* ,24)) = Р(р(рэ(т1,--- ‚т ))) = о(р1)Р(рә(т,--- ,2)) = 0(pi)o (pi) P: , £n). 
So o(p1 орг) = o(pi)o(p2). п 
2. 


Proof. To see (c) is true, we suppose t interchange ip and јо. Without loss of generality, we assume io < jo. 
Then 


1 


Р( ту, ,т„)) Р(ш\, + ‚ты Tio En) 


= (28-5) [[ 6-2) 
i<j,(ij)#(iojo) 
= -[It-2) 
i<j 
= -P(21,-- 2а). 


So o(t) = —1. 

To see (d) is true, note formula (9) is equivalent to id = ty o ---tı орт. Acting these operations on 
(1,-++,n), we have (1, --- ут) = tp o---oti(p-! (1),--- ,p71(n)). Then the problem is reduced to proving 
that a sequence of transpositions can sort an array of numbers into ascending order. There are many ways 
to achieve that. For example, we can let tı be the transposition that interchanges p^! (1) and p^! (io), where 
ig satisfies p~!(i9) = 1. That is, Ё puts 1 in the first position of the sequence. Then we let tz be the 
transposition that puts 2 to the second position. We continue this procedure until we sort out the whole 
sequence. This shows sorting can be accomplished by a sequence of transpositions. о 


3. 


Proof. For any transposition Ё, we have tot = id. So if p = t+ o --- o tiy we can get another decomposition 
p=t,o---ot,otot. This shows the decomposition is not unique. 

Suppose the permutation p has two different decompositions into transpositions: р = ty o- oti = 
1^, 0---0t1. By formula (7), part (b) and formula (8), о(р) = (—1)* = (-1)™. So k — m is an even number. 


m 
This shows the parity of the member of factors is unique. [m] 


4. 
Proof. To verify Property (ii), note for any index j, a, Є К, we have 
D(ai,--- аа; + Ваз, an) 

У о(ђар --- (025, + Bap) apan 


Sle Papa 725,5 ар, + Во(р)аьа "арза 
oD(n,--- a, a5)  BD(m, aj, sts). 


To verify Property (iii), note ¢p,1---€p,n is non-zero if and only if p; = i for any 1 <i < n. In this case 
the product is 1. 
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To verify Property (iv), note for any i  j, if we denote by t the transposition that interchanges i and j, 
then p++ pot is a one-to-one and onto map from the set of all permutations to itself. Therefore, we have 


D(a,- ,а а, 


ал) 

= Leaps tetas 

= У оро дааа ба аран pru 
= Lele даира ара ptas 


= (HV) È о(фам Фу салыс Agan 
= —D(ay,-+- ‚ау, 977,0). 


o 
5. 
Proof. By property (iv), D(ai,-++ ,ai,+++ ,@i,+++ ,@n) = —D(ai,++- ,ai,+++ ,Qi,*** ,ал). So add to both 
sides of the equations D(ai,--- ,ai,::: ,@i,-*+ ал) , we have 2D(a1,--- ,ai,-++ ,Qi,*** ,аһ) = 0. If the 
character of the field K is not two, we can conclude D(ai,--- ,ai,-++ ‚а, ал) = 0. о 


Remark 5. This exercise and Exercise 5.4 together show formula (16) is equivalent to Properties (i)-(iit), 
provided the character of К is not two. Therefore, for К = R or C, we can either use (16) or properties 
(i)-(iti) as the definition of determinant. 


6. 


Proof. If two column vectors a; and aj (i 7 j) of Ai; are equal, we have В = [2]. So C(Ai1) = 0 and 
у 
property (i) is satisfied. Since any linear operation on a column vector a; of Ai; can be naturally extended 


to b , property (ii) is also satisfied. Finally, we note when Aj; = I(n—1)x(n=1)> р xi = хл. So 
à 

property (iii) is satisfied. 

vis 


Proof. We first move the j-th column to the position of the first column. This can be done by interchanging 

neighboring columns (у — 1) times. The determinant of the resulted matrix A; is (—1)/-!detA. Then we 

move the i-th row to the position of the first row. This can be done by interchanging neighboring rows 

(i — 1) times. The resulted matrix Аз has a determinant equal to (—1)^!detA; — (—1)*2detA. On the 

other hand, Az has the form of б h ). By Lemma 4, we have detA,, = detAy = (—1)*¥detA. So 
dj. 

detA = (—1)*det A;;. o 


Remark 6. Rigorously speaking, we only proved that swapping two neighboring columns will give a minus 
sign to the determinant (Property (iv)), but we haven't proved this property for neighboring rows. This can 
be made rigorous by using det A = det АТ (Exercise 8 of this chapter). 


8. 


Proof. We first show for any permutation p, о(р) = o(p-!). Indeed, by formula (7)(b), we have 1 = o(id) = 
о(ро р!) = o(p)o(p-1). By formula (7)(a), we conclude o(p) = o(p-1). Second, we denote by b; the 


15 


(i, j)-th entry of AT. Then bj; = ау. By formula (16) and the fact that p+ p~! is a one-to-one and onto 
map from the set of all permutations to itself, we have 


detA” = S°o(p)bpi1---bpan 
= E оа anp, 
= J o(p aoc 77° pep) npn 
= Уор Dass i tpn) Pn 


= Msi» Dass ap-i(n)n 
det A. 


9. 


Proof. By Exercise 2, it suffices to prove the property for transpositions. Suppose p interchanges i1, i; and 
q interchanges jı, јә. Denote by P and Q the corresponding permutation matrices, respectively. Then for 
any т = (тү, ,2,)7 € R^, we have (5;; is the Kronecker sign) 


т ifi=i 
(Рт), = 3 Pjr; =) бут = m4 ifi= iz 
zi otherwise. 


This shows the action of P on any column vector x performs the permutation p on the components of x. 
Similarly, we have 


aj, ifi=j 
(Qr) = fz} ifi-j 
a; otherwise. 


Since (PQ)(r) = P(Q(r)), the action of matrix PQ on т performs first the permutation q and then the 
permutation p on the components of т. Therefore, the permutation matrix associated with p o q is the 
product of P and Q. o 


10. 
Proof. 
s mo mon в. m А 
tr(AB) = УАВ): = Уан = Y Уа = УУ ан = УВА): = (ВА), 
= jai dei e jul je pri 
where the third equality is obtained by interchanging the names of the indices i, j. [m] 
п. 


Proof. 


(ААТ) = улат) 


E Ain = Y aay = 
Е: 3773 ij 


12. 


Proof. Apply Laplace expansion of a determinant according to its columns (Theorem 6). п 
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13. 


Proof. Apply Laplace expansion of a determinant according to its columns (Theorem 6) and work by induc- 
tion. п 


14. 


Proof. Denote by M(n) the number of multiplications needed to evaluate det A of an n x n matrix А by 
using Gaussian elimination to bring it into upper triangular form. To use the first row to eliminate ал, 
азі, «++, ап, we need n(n — 1) multiplications. So M(n) = n(n — 1) + M(n — 1) with M(1) = 0. So 
M(n) = У K(k -1) = ща+1@п+1) zt заны) = (nni). o 


15. 


Proof. Denote by M (n) the number of multiplications needed to evaluate the determinant of an n x n matrix 
by formula (16). Then M(n) = nM(n — 1). So M(n) = nl. п 


16. 


Proof. We apply Laplace expansion of a determinant according to its columns (‘Theorem 6): 


sb ve f b c b 
а [а e f adet|^ J|-ddet|? S| +gdet|? ^ 
А h i h i e f 
g h i 
a(ie — fh) — d(ib — ch) 4- g(bf — ce) 
aei + bfg + саһ — gec — afh — idb. 


1 


o 

6 Spectral Theory 
1. 
Proof. fsz = aj? /У5 = 2178309. а 
2. (а) 
Proof. Denote by hj the eigenvector corresponding to the eigenvalue aj. For any h € C", there exist 
01-7120. € C such that h = ¥);0;h;. So ANh = Y^; 6;aj һу. Define b = max{|ay|,--» , |an|}. Then for any 
1< k x n, (Ah) = |X; 850) (һу) < b" Y; Ө) — 0, ав N — оо, since 0 < b < 1. This shows 
ANh > 0 as N — оо. a 

(b) 


Proof. We use the same notation as in part (a). Since h # 0, there exists some ko so that hig = JD; 0;(hj)ro # 
0. Then |(ANA)io| = |57, буа (hj)ro|. Define bı = maxicicn{lail : ® # 0, (А) #0}. Then by > 1 and 


hence САХА) | = [bul |S 6:55 (ы)ы| = оо as N — оо. a 
А 

3. 

Proof. The verification is straightforward. o 

4. 


Proof. Formula (24) gives us Af = af + h, which is formula (25) when N = 1. Suppose (25) holds for any 
n < N, then AN*! f = A(ANf) = A(a™ f + NaN-!h) = a Af + NaN—1 Ah = a (af + h) + NaN-!ah = 
aN+1 f + (N +1)ah. So (25) also holds for N + 1. By induction, (25) holds for any N € N. о 
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5. 


Proof. Suppose q(s) = У," bis, then by formula (25), q(A)f = У" БА = У" o bi(a f + iaith) = 
(Sho bia) f + (Xi Фа? T)h = q(a)f + q'(a)h. п 


6. 
Proof. By Lemma 9, Np,...p, = Np, Ө Np... = Np, € (Npa € Nps---pa) = Np, ® Npa ® Np, = °° = 
Мы ® Npa O +++ ® Npr- o 
7. 
Proof. For any т € Na(a), we have (A — aI)*(Az) = (A — al)**1z + a(A — аГ)%т = 0. So Ar € №(а). O 


8. 


Proof. A number is an eigenvalue of A if and only if it’s a root of the characteristic polynomial pa. So 
pa(s) can necessarily be written as pa(s) = []\(s— a;)™ with each m; a positive integer (i = 1,--- ,k). We 
have shown in the text that рд is a multiple of m4, so we can assume m4(s) = IG — а;)'* with each r; 
satisfying 0 < r; < m; (i = 1,--- ,k). We argue r; = d; for any 1 < i € k. 

Indeed, if for some j, ту < dj, we can find x € N4,(aj) V N,,(a;) with т 7 0. Define q(s) = Iia; - 
a;)", then by Corollary 10 т can be uniquely decomposed into 2’ + z" with z' € № and т” € N,,(aj). We 
have 0 = (A — ajI)!/z = (А —ajI)%2’ + 0. So z' € №, N №, (а;) = (0). This implies т = x” € №, ,(а;). 
Contradiction. Therefore, r; > d; for any 1 € i < k. 

On the other hand, if for some j, rj > dj, then m', = IT. inal = ai)" - (s — aj) satisfies deg(m^,) < 
deg(ma) and Nm, = Nn (a1) © ++: € Na, (aj) © --: 6 Ny, (ак) = Na, (m) © +++ 6 Na, (ак) = C" by ri 2 di 
(1 € i € К), which is proved above. So тд cannot be the minimal polynomial. Contradiction. 

Combined, we conclude ma(s) = [TE (s — ai)". Oo 


Remark 7. Minimal polynomial is defined from the algebraic point of view as the generator of {polynomial р: 
p(A) =0}. So the powers of its linear factors are given algebraically. Meanwhile, the index of an eigenvalue 
is defined from the geometric point of view. Theorem 11 says they are equal: 


Remark 8. An alternative proof goes as follows. By Theorem 5, Corollary 10, and the definition of index, 
we have (we assume the characteristic polynomial p4(s) = П;_,(8 — a;)”) 


k 
C" = Np, = (DN, (а). 
ј=1 


(A — aj1) із nilpotent on N,,, (aj) with indez dj. So A|w,, (а;) has minimal polynomial (s — a5)? and the 
minimal polynomial of A on С" is therefore TE - aj)^. 
Remark 9. As a corollary, we claim A can be diagonalized over the field F if and only if its minimal 
polynomial can be decomposed into the product of distinct polynomials of degree 1 over the field F. Indeed, 
by the uniqueness of minimal polynomial, we have 
ma is the product of distinct polynomials of degree 1 
= dh=.-=d=1 


k 
= œ= Mla) 
j=1 
<= A can be diagonalized, where the basis is US_,{ej1,--- eji) with {ej1, -+> eji) a basis for Ni(aj) 


Remark 10. The following proposition gives an elementary proof to Lemma 10, Chapter 9. 
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Proposition 1 (Geometric multiplicity and algebraic multiplicity). Let A be an n x n matriz over 
a field Е and a an eigenvalue of A. If m is the multiplicity of a as a root of the characteristic polynomial of 
A, then dimN,(a) € m, where № (а) = null space of (A — al) is the space of eigenvectors pertaining to a. 

m is called the algebraic multiplicity of a and dimN;(a) is called the geometric multiplicity of a. So this 
theorem says “geometric multiplicity < algebraic multiplicity”. 


Proof. Let vı,- ,v, be a basis of № (a) and extend it to a basis of F^: vi, ---, Us, ш, 5, Up. Define 
U = (vi, „аш, Ur). Then 
ОАО = U^A(u,-- vs use ur) 
U^ (av, --- , avs, Aur, -+ , Aur) 
(aU7 wi, aU vU Aui, UT Aur). 


Because U-!U = I, U-!AU = [ee al and det(AI — A) = det(AI — U^! AU) = (A — а)* det(AI — C). 


боз < т. o 


Remark 11. It’s a good place to summarize the relationship between index, algebraic multiplicity, geometric 
multiplicity, and the dimension of the space of generalized eigenvectors pertaining to a given eigenvalue. We 
have the following sequence of results: 


Proposition 2 (Index and algebraic multiplicity). Let A be an n x n matriz over a field Е and a an 
eigenvalue of A. Denote by m the multiplicity of a as a root of the characteristic polynomial pa and by d 
the index of a, then d < m. 


Proof. Let q(s) = pa(s)/(s—a)™. Then С" = Npa = Ng Ф Nm(a). For any v € Nyn+i(a), v can be uniquely 
written as v =v +v” with v' € №, and v" € N,,(a). Then v' = v—v" € Nm+1(a)NN;. Similar to the second 
part of the proof of Lemma 9, we can show v/ = 0. So v =v" € Nm(a). This shows №,„ (а) = N,,,(a) and 
hence d € m. o 


Proposition 3 (Algebraic multiplicity and the dimension of the space of generalized eigenvec- 
tors). Let A be an n x n matriz over a field F and a an eigenvalue of A. Denote by m the multiplicity of a 
as a root of the characteristic polynomial pa and by k the dimension of the space of generalized eigenvectors 
of A pertaining to a, then k = т. 


Proof. See Theorem 11 of Chapter 9. [m] 


Summary: numbers associated with an eigenvalue. Let A be an n x n matriz over a field F and 
a an eigenvalue of A. Then we have 


IA 


tric multiplicit; А М у "n 
ue vio mulivlicity of a dim. of the space of generalized eigenvectors pertaining to a 


index of a 


Ш 


algebraic multiplicity of a. 


9. 
Proof. The extension is straightforward as the key feature of the proof, “В maps NO) into NO)”, remains 
the same regardless of the number of linear maps, as far as they commute pairwise. d 
10. 

Proof. For any i € (1,--- ,n), by Theorem 17, (ту) = 0 for any j # i. Since £1, ---, т, span the whole 
space and l; # 0, we must have(l;, ;) # 0, i = 1,--- ‚п. This proves (a) of Theorem 18. For (b), we note if 
т = Ya буту, then (1,2) = k(li zi). So ki = (lis 2)/ (s). п 


1For the last equality, see, for example, Munkres [5], page 24, Problem 6. 
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11. (a) 


Proof. The matrix is symmetric, so it's equal to its transpose and the eigenvectors are the same: for eigenvalue 
a; = 15, the eigenvector is hy = Е Т for eigenvalue аз = 1/5, the eigenvector is hy = [. ] | a 
à 2 


(b) 


Proof. We note (hi, h;) = 1+ аў = 5£Y5 and (hj, h2) = 1 + a3 = 535. For 2 = 1. we have (h;,) = а 
and (hg, т) = a. So using formula (44) and (45), т = сүһү + coho with 


5+Vv5 5- v5 
а=а/—5 =1/V5, о = 0/9 = -M VS. 
This agrees with the expansion obtained in Example 2. a 


12. 


Proof. The transpose of the matrix has the same eigenvalues a; = 2, a? = 5. Solving the equation 


[> | Й =2 Ё. we have h = [1 —1]. Solving the equation Ё АЦ] =5 Ji we have lp = [1 2]. 

Then it's easy to calculate (l1, h1) = 3, (lı, h2) = 0, (l2, h1) = 0, and (lz, h2) = 3. 

13. 

Proof. 
- -1-A -1-X 

det(AI— A) = det A А+1 -1-A | = -[(A+1) =A- (АР) = (A+1)? (4-2). 
-1 -1 A 

So the eigenvalues of A are —1 and 2, and the eigenvalue 2 has a multiplicity of 2. o 


7 Euclidean Structure 


1. 
Proof. By letting y = үү, we get [т] < maxjy-i(,y). By Schwartz Inequality, maxy,)=1(7,y) < [т]. 
Combined, we must have |x| = max),)=1(x, y). [m] 
2. 


Proof. Vr,y and suppose their decomposition are ту + 72, yi + y», respectively. Неге zı,yı € Y and 
72,5 € Y+. Then (Pyy, x) = (y, Рут) = (yı +2,21) = (и, т) = (1,2) = (Pyy, £). By the arbitrariness 


of т and y, Py = Р}. o 

3. 

Proof. Under the reflection across the plane {(z1, 12, £3) : тз = 0}, point (21,22,23) will be mapped to 

(x1, тә, —73). So the corresponding matrix is [0 1 à ) whose determinant is —1. a 
00 -1 
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Proof. Suppose the plane L is determined by the equation Ar + By + Cz = D. For any point т = 
(x1, 22,23)’ € R, we first find y = (y1,y2,y3)’ € L such that the line segment zy L Г. Then y must 
satisfy the following equations 


Ay + By + Cys = D 
(n — 21, y2 — 22, ys — тз) = К(А, B,C) 


where k is some constant. Solving the equations gives us k = 202258225022) and 


u Ti A Ti 1 А2 AB AC] [n D A 
р | = |z2| +k |B| = |2| ——m——|A4AB B? ВС| |22| += |В 
pe ua C Pa A? + B? + C? CA CB С | |z A? + В? +С? C 


So the symmetric point z = (21, 22, z3)/ of x with respect to L is given by 


а nm) [m] [m j А2 AB AC] [n áp А 
zo} = 2 || – || = | -————5|AB B? ВС| |2| +- |В 
2 2|- [ni A&*B*O lon св |||) Prete |o 

i АЗ ВЗ +С2 -2AB -24C тү 55 A 

= ~~ | -24B 4?-B?+0? — BO | |n o +22 |B 

AP + В? +С -2CA -20B  A?4+B?-C?| |z| +8 +0 |o 


To make the reflection R a linear mapping, it’s necessary and sufficient that D = 0. So the problem’s 
statement should be corrected to “let R be reflection across any plane in R? that contains the origin". Then 


1 —A? + В? +С? —2AB -2АС 
В = 5—5 —2AB А? — В? +С? -2BC 
P. 7 -2CB A? #B?-C? 
R is symmetric, so R* = R and by plain calculation, we have R*R = R? = I. By Theorem 12, R is an 
isometry. п 
5. 
Proof. Suppose M is an n х n orthogonal matrix. Let r},--- ‚т, be its column vectors. Then 
n nri тиў c nnm 
EMA 2 |...) rt, Sry) КЛ... f£ 
т таг) Тыт} с Tag. 
So M is orthogonal if and only if rir? = 6;; (1 < i,j € n). о 
6. 
Proof. Note |a;;| = sign(a;;) - ef Ae;, where ер is the column vector that has 1 as the k-th entry and 0 
elsewhere. Then we apply (ii) of Theorem 13. o 
T. 
Proof. See the solution in the textbook. o 
8. 


Proof. For any x,y € X and a € C, 0 < |x — ау] = [2] — 2Ве(т,ау) + |а[?у]?. Let a = $52 (assume 
y # 0), then we have 


lul? ш? ° 
which gives after simplification |(z, y)| < 12111. a 


re неа Een} 4 Ie: 
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9. 


Proof. Proofs are the same as the ones for the real Euclidean space. п 
10. 
Proof. Proof is the same as the one for real Euclidean space. п 
11. 


Proof. If M is a unitary map, then by parallelogram law, M preserves inner product. So Уг, y, (т, M* Му) = 
(Mz, My) = (x,y). Since т is arbitrary, M*My = y, Yy € X. So М+М = I. Conversely, if M*M = I, 


(2,2) = (т, M* Mz) = (Mz, Mz). So M is an isometry. п 
12. 
Proof. (М7їт‚,М-їт) = (M(M-!z), М(М-їт)) = (2,2). (Mz, Mz) = (т,т) = (M*Mz, M*Mz). By 
Ru = X, (y, y) = (M*y, M*y), Vy € X. So М-! and M* are both unitary. п 
13. 


Proof. If M, N are two unitary maps, then (MN)*(MN) = N*M*MN = N*N = Г. So the set of unitary 
maps is closed under multiplication. Exercise 12 shows that each unitary map has a unitary inverse. So the 
set of unitary maps is a group under multiplication. о 


14. 


Proof. By Exercise 8 of Chapter 5, det M* = det M^ = det M. So by M*M = I, we have 1 = det M* det М = 
|det Мђ2, i.e. |det M| = 1. п 


15. 


Proof. (Mf, Mf) = [1 Mf(s)Mf(s)ds = J*, m(s)f(s)m(s)F(s)ds = f^, |т(в)?|/(в)|°4в = (f, f). This 
shows M is unitary. о 


16. 
Proof. The proof is very similar to that of real case, so we omit the details. Note we need the complex 
version of Schwartz inequality (Exercise 8). о 
17. 
Proof. We have 

aj n 

(АА*);; = [ain -> pain] |+ = Manag. 

йм] к= 
So (АА*)н = Xx, |ы}? and tr(AA*) = Z; 10. o 
18. 
Proof. This is straightforward from the result of Exercise 17. o 
19. 


Proof. Suppose Ау and А» are two eigenvalues of A. Then by Theorem 3 of Chapter 6, Àj + Аз = trA = 4 
and АА» = det А = 3. Solving the equations gives us А = 1, А = 3. By formula (46), |A| > 3. According 
to formula (51), we have |А < VI? +2 + 32 = v/14. Combined, we have 3 < |A] < V14 ~ 3.7417. — D 


20. (i) 
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Proof. For any a, оз € F, we have 


(w(oizi--a272,9),z) = det(a, x1 + a2r2,y,z) = o1 det(zi, y, 2) + az det (тә, y, 2) 
ол (w(zi, y), 2) + аз(ш(тэ, у), 2) = (атш(т1, y) + азш(тэ, y), z). 


Since z is arbitrary, we necessarily have ш(оџту + a272,y) = o1w(zi,y) + азш(тә,у). Similarly, we can 
prove w(z, етуу + a2y2) = атш(т, уу) + азш(т, у»). Combined, we have proved w is a bilinear function of т 
and y. o 


(ii) 
Proof. We note 
(w(x, y), 2) = det(, y, z) = — det(y, x, z) = —(w(y, £), 2) = (—w(y, £), 2). 
By the arbitrariness of z, we conclude w(z,y) = —w(y,2), ie. yx £ = —z x y. a 
(ш) 


Proof. Since (w(r, y), £) = det(x, y, x) = 0 and (w(z, y), y) = det(x,y,y) = 0, x х y is orthogonal to both т 
апа у. o 


(iv) 
Proof. We suppose every vector is in column form and R is the matrix that represents a rotation. Then 
(Rz x Ry, z) = det(Re, Ry, z) = (det R) - det(x, y, R^!) 


and 
(R(z x у), 2) = (В(т x y))z = (т x y) RT z = (т x y, RT z) = det(r, y, ИТ). 


A rotation is isometric, so АТ = R^! and det R = +1. Combing the above two equations gives us +(Rx x 
Ry, z) = (R(a x y), 2). Since z is arbitrary, we must have --Rz x Ry = R(r x y). o 


(v) 


Proof. In the equation det(r, y, 2) = (x x y, 2), we set z = т x y. Since the geometrical meaning of det(r, у, 2) 
is the signed volume of a parallelogram determined by т, y, 2, and since 2 = т x y is perpendicular to x and 
y, we have det(x,y, 2) = + |2112 sin 0[21, where 0 is the angle between x and y. Then by (т x y, 2) = |г[?, 
we conclude | x y| = |z} = 11101 sind. o 


(vi) 
10 0 1] [o] [o 
1-det|0 1 0| = (ох |, fof). 
001 oj jo} |i 


roof. 
| [o] fa 1] fo 
So [o| x 1| = |b|. By part (iii), we necessarily have a = b = 0. Therefore, we can conclude |0| x |1| = 
1 oj |o 
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Proof. By Exercise 16 of Chapter 5, 


a d g 
det |b e h 


cfi 


a bc 
de|d e f 

ghi 
aci + bfg + cdh — gec — hfa — idb 
(bf — ec)g + (cd — fa)h + (ae — db)i 


g 
[/—сє cd—af ае H 


i 


So we have 
a d bf —ce 
b| x Je| = |ed—af| . 
c f ae — bd 
o 
21. 
Proof. 


lu tol? + Ju— v? (u+v,ut+v) + (и — vu о) = (u,u +v) +(v,u+v) + (u,u — v) — (v,u — v) 


(и, и) + (u,v) + (v, и) + (v, v) + (и, и) — (u,v) — (v, u) + (v, v) = 2]u? + 2]vf?. 


o 


8 Spectral Theory of Self-Adjoint Mappings of a Euclidean Space 
into Itself 


The following result will help us understand some details in the proof of Theorem 4’ (page 108, “It follows 
from this easily that we may choose an orthonormal basis consisting of real eigenvectors in each eigenspace 
Na”) 


Proposition 4. Let X be a conjugate invariant subspace of C". (i.e. X is invariant under conjugate opera- 
tion). Then we can find a basis of X consisting of real vectors. 


Proof. We work by induction. First, assume dim X = 1. Vv € X with v 4 0, we must have Rev € X and 
Imv € X. At least one of them is non-zero and can be taken as a basis. Suppose for all conjugate invariant 
subspaces with dimension no more than К the claim is true. Let dim X = k +1. Vv € X with v # 0. If Rev 
and Imv are (complex) linearly dependent, there must exist c € C and vo € R" such that v = cvo, and we let 
Y = span{vo}; if Rev and Imv are (complex) linearly independent, we let Y = span{v, v) = span{Rev, Imv}. 
In either case, Y is conjugate invariant. Let Y+ = {x € X : Egy zig; = 0, Vy = (Y1, ys)! € Y). Then 
clearly, X = Y @Y+ and Y- is also conjugate invariant. By assumption, we can choose a basis of Y+ 


consisting exclusively of real vectors. Combined with the real basis of Y, we get a real basis of X. O 
n 
Proof. 
м+м \ 1 PENAS 
(© a 2) = (в Mz) + (2, M*a)] = ite. Mr) + (Mz, )] = ite. Mz) + (@,Ma)| = Re(z, Ма). 


o 


Proof. We prove p} + po = max,(s)>odimS. p- + ро = max4(s)<odimS can be proved similarly. We shall 


use representation (11) for q in terms of the coordinates 21, ---, Zn; suppose we label them so that di, ··-, 
d, are nonnegative where р = p+ + ро, and the rest are negative. Define the subspace S; to consist of 
all vectors for which 2-41 = --- = z, = 0. Clearly, dimS; = p and q is nonnegative on 51. This shows 
рь + Po = p < max,s)>9dimS. If < holds, there must exist a subspace S> such that q($2) > 0 and 
dimS2 > p = py + po. Define P: $9 > S := {z : 241 = Zp42 = +++ = Zn = 0} by P(z) = (21,5 25,0, ,0). 
Since dimS4 > р = dimSs, there exists some z € Sz such that z # 0 and P(z) = 0. This implies 
а =з 2р = 0. So q(z) = Dy diz? + У die? = Diy die? < 0, contradiction. Therefore, 
our assumption is not true and < cannot hold. o 


4. 
Proof. Write M in the column form M = [c1,* ++ , c4] and multiply M to both sides for formula (24)’, we get 


м 4 o 
0 X 0 
HM = [Hey,-+- Hea] = MD = [en sen] | 22 77 | ра, Ае), 
0: 07552, А 


where Л, +++, An are eigenvalues of M, including multiplicity. So we have Hc; = Aici, i = 1,-++ ,n. This 
shows the columns of M are eigenvectors of H. D 


5. 


Proof. The essence of the generalization can be summarized as follows: (x,y) = (x, My) is an inner product 
and M-!H is self-adjoint under this new inner product, hence all the previous results apply. 

Indeed, (x,y) is a bilinear function of x and y; it is symmetric since M is self-adjoint; and it is positive 
since M is positive. Combined, we can conclude (x,y) is an inner product. 

Because M is positive, Mx = 0 has a unique solution 0. So M~! exists. Define U = M-!H and we 
check U is self-adjoint under the new inner product (-,-). Indeed, Wa, y € X, 


(Ux, y) = (Ux, My) = (M Hz, My) = (Hz, y) = (т, Hy) = (т, MM™ Hy) = (z, MUy) = (z, Uy). 


Applying the second proof of Theorem 4, with (-,-) replaced by (-,:) and H replaced by M-!H, we can 


verify claims (a)-(d) are true. o 
6. 
Proof. This is just Theorem 4 with (-,-) replaced by (-,-) and H replaced by МЇН, where (-,-) is defined 
in the solution of Exercise 5. a 
T. 
Proof. This is just Theorem 11 with (-,-) replaced by (:,:) and H replaced by M~1H, where (-,-) is defined 
in the solution of Exercise 5. a 
8. 


Proof. Under the new inner product (-,-) = (-,M-), U = M-!H is selfadjoint. By Theorem 4, all the 
eigenvalues of M-1H are real. If Н is positive and M-! Hz = Ат, then A(z, £) = (т, M-! Hz) = (z, Hz) > 0 
for т # 0, which implies А > 0. So under the condition that Н is positive, all eigenvalues of M-!H are 
positive. o 


10. 
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Proof. By Theorem 8, we can assume N has an orthonormal basis v1,--+ , Un consisting of genuine eigen- 
vectors. We assume the eigenvalue corresponding to v; is nj. Then by letting т = vj, j = 1,---,n and 
by the definition of |N], we can conclude |N] > max Meanwhile, Vr € X with |x| = 1, there exist 
41,::: à € C, so that Ð |а; = 1 and т = } ajvj. So 


INe] 
Т = от = VY IP < pax nj ajl? = maxing. 


This implies |N| < max |n;|. Combined, we can conclude |N] = max [n;]. п 


Remark 12. Compare the above result with formula (48) and Theorem 18 of Chapter 7. 


1X. 
Proof. |S(a1,-*+ ,a5)| = |(a5,a1,:** ,@n—1)| = |(а1,--- ,@n)|. So S is an isometry in the Euclidean norm. 
To determine the eigenvalues and eigenvectors of S, note under the canonical basis ¢1,-+~ , €n, S corresponds 
to the matrix 

б: ф Q — 0 

1 9 °@ «70-0 

A=|0 1 0-- о of, 
ооо 1 0 


whose characteristic polynomial is p(s) = |A—sI| = (—s)"--(—1)"*!. So the eigenvalues of S are the solutions 
to the equation s" = 1, ie. Ay = e ,k-1.n. Solve the equation Sx, = Акт, we can obtain the gen- 
eral solution as ть = (Ag ^, А172, +++ , Ak, 1)'. After normalization, we have ть = WORST, Xs D. 
Therefore, for i # j, 


12. (i) 


Proof. A*A = B 3] М Al = р MES has eigenvalués 74-/40. By Theorem 13, |A] = \/7 + V ~ 


3.65. o 
(ii) 

Proof. This is consistent with the estimate obtained in Exercise 19 of Chapter 7: 3 < | Al < 3.7417. o 

13. 


-10 1 
13, the norm of the matrix is approximately ү 13.3523 ~ 3.65. Qo 


1 Yu 5 6 -1 
Proof. | 0 3 b З | = 6 9 0 |, which has eigenvalues 0, 1.6477, and 13.3523 By Theorem 
=1 0 
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9 Calculus of Vector- and Matrix- Valued Functions 


1. 


Proof. Following the hint, note # (x(t), у) = (2(£), y)--(z(t), 9) = 0. So (x(t), y) is a constant by fundamental 
lemma for scalar valued functions. Therefore (x(t) — (0), y) = 0, Vy € K”. This implies x(t) = z(0. О 


2. 
Proof. A` (£)A(t) = I. So 0 = 2 [A7(0)4(0)] = 247! (t) - Alt) + A7 (t)À(t) and 24-00) = 4 А-1(%). 
A(t)A-! (t) = —A-1(t) À(t) A7! (t). H 
3. 


ý Lx? if n is even 
0 1" Ју ү 
1 0) > ( | if n is odd. 


Therefore, we have 


SI/01" e Lo v 1 0 IN ecce! 
ехр{А+ В} = Ya 0) =E etd aap k ) - 2 


o 
4. 
Proof. For any = > 0, there exists M » 0, so that Ym > M, sup, |En(t) — F(t)| < £. So Ym > M, Vt, h, 
1 
[Шве 200-0) 
1 ph poe 
= |5 Í [Bni(s) — F()ds + = F(s)ds — F(t) 
t t 
th AE р! th 
LV. а) Eolas, |а F(s)ds — E(f) 
Qo 
< e] F(s)ds — F(t)|. 
hh 
Under the assumption that F is continuous, we have 
‚|1 : 1 
lim [шее +h) — Е(0)] – ғо = lim lim jine +h) - Em(t)] — rol 
1 г 
< e+ jim Hi F(s)ds — F(t)| =e. 
Since є is arbitrary, we must have lim, 1 [E(t + h) — E(t)] = F(t). [m] 


5. 
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Proof. By formula (12), Em (t) = У", D'S} KA (0) À(t) A777 (t). So for m and n with m < n, 


. . n i k-i-1 n k-1 
мово < у; MOA OI у Rawr 


kimi ico ьт ico 
S Mok- 
= > T- KOM Aol- = Ollen (AEI — em(IA@D] > 0 
кн. 
as m,n > оо. This shows (Em(t))2°_; is a Cauchy sequence, hence convergent. a 
6. 
Proof. Apply formula (10) to Y (t) = е^, we have 2 log det Y (t) = tr(e~4*e**A) = trA. Integrating from 0 
to t, we get log det Y(t) — log det Y (0) = А. So det Y(t) = eT. In particular, det e^ = ef. о 
7 


Proof. Without loss of generality, we can assume А is a Jordan matrix. Then e^ is an upper triangular 


matrix and its entries on the diagonal line have the form е“, where a is an eigenvalue of A. So all eigenvalues 
of e^ are the exponentials of eigenvalues of A. o 


8. (a) 


Proof. The total number of “free” entries is 7“). The entries on the diagonal line must be real. So the 
dimension is G2 x 2—n=n?. o 


(b) 


Proof. Similar to the argument in the text, the total number of complex parameters that determine the 
eigenvectors is (n — 1) ------2— aed) —1. This is equivalent to n(n — 1) — 2 real parameters. The number 
of distinct (real) eigenvalues is n — 1. So the dimension = n? -n 2 + п - 12 n?— 3. a 


9. 
Proof. See the Matlab program aoc.m. 


function aoc 


ЖАОС illustrates the avoidance-of-crossing phenomenon 

4 of the neighboring eigenvalues of a continuous 

% symmetric matrix. This is Exercise 9, Chapter 9 

4 of the textbook, Linear Algebra and Its Applications, 
X 2nd Edition, by Peter Lax. 


4 Initialize global variables 
matrixSize = 10; 
lowerBound = 0.01; %lower bound of t’s range 
upperBound = 3; %upper bound of t’s range 
stepSize - 0.1; 

= lowerBound:stepSize:upperBound; 


% Generate random symmetric matrix 
tempi = rand(matrixSize); 

temp2 - rand(matrixSize); 
tempi+temp1’; 

В = temp2+temp2’ ; 
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% Initialize eigenvalue matrix to zeros; 
% use each column to store eigenvalues for 
% a given parameter 
eigenval = zeros(matrixSize,numel(t)); 
for i = 1:numel(t) 
eigenval(:,i) = eig(B+t(i)*M); 
end 


% Plot eigenvalues according to values of parameter 
hold off; 
disp(['There are ', num2str(matrixSize), ’ eigenvalue curves.’]); 
displ 7); 
for j = 1:matrixSize 
disp(['Eigenvalue curve No. ’, num2str(j),'. Press ENTER to continue...’]); 
plot(t, eigenval(j,:)); 
xlabelCt?); 
ylabel('eigenvalues'); 
title('Matlab illustration of Avoidance of Crossing’); 
hold on; 
pause; 
end 
hold off; 


10 Matrix Inequalities 
1. 


Proof. Suppose H has k distinct eigenvalues Ai, ---, Ax. Denote by X; the subspace consisting of eigenvectors 
of H pertaining to the eigenvalue Ау. Then Н can be represented as Н = Si AjP; were P; is the projection 
to Let A be a positive square root of H, we claim A has to be Sha МЗ; ^j. Indeed, if a is an eigenvalue 
of A and т is an eigenvector of A pertaining to a, then o? is an eigenvalue of Н and т is an eigenvector of 
Н pertaining to a?. So we can assume A has m distinct eigenvalues a1, :--, аа (m € К) with a; = VÀ; 
(1 € i < m). Denote by Y; the subspace consisting of eigenvectors of A pertaining to oj. Then Y; С Xj. 
Since Н = (z, Y; = ӨХ, we must have т = Ё and Ү; = X;, otherwise at lease one of the < in the 
sequence of inequalities dimH = У", dimY; < У", dimX; < УЖ, dimX; = dimH is <, contradiction. So 
A can be uniquely represented as A = Y V/A; P;, the same as VH defined in formula (6). [1 


2. 


Proposition 5. (i) The identity I is nonnegative. (ii) If М and N are nonnegative, so is their sum 
М + №, as well as aM for any nonnegative number a. (iii) If H is nonnegative and Q is invertible, we have 
Q*HQ > 0. (iv) Н is nonnegative if and only if all its eigenvalues are nonnegative. (vi) Every nonnegative 
mapping has a nonnegative square root, uniquely determined. 


Proof. (i) and (ii) are obvious. For part (iii), we write the quadratic form associated with Q*HQ as 
(r. Q* HQz) = (От, HQz) = (y. Hy) > 0, 


where y = Qz. For part (iv), by the selfadjointness of H, there exists an orthogonal basis of eigenvectors. 
Denote these by hj and the corresponding eigenvalue by aj. Then any vector x can be expressed as a linear 


combination of the hj’s: т = У), zjh;. So (т, Hz) = X; (zihi, zjajhj) = У" ау|ту|?. From the formula 
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it is clear that (x, Hz) > 0 for any z if and only if a; > 0, Vj. For part (vi), the proof is similar to that of 
positive mappings and we omit the lengthy proof. Cf. also solution to Exercise 10.1. п 


8: 
Proof. Let A be a mapping that maps the vector (0, 1)' to (0,@»)/ with a2 > 0 sufficiently small and (1,0) 
to (01,0) with a; > 0 sufficiently large. Let В be a mapping that maps the vector (1, 1)’ to (A1, A1)! with 


№ > 0 sufficiently small and (—1, 1)! to (А, 2)’ with А» > 0 sufficiently large. Then both A and В are 
positive mappings, and we can find т between (1, 1)' and (0, 1)' so that (Ат, Br) < 0. By the analysis in 


the paragraph below formula (14)/, AB + BA is not positive. More precisely, we have A = m M ) and 
a 
La (X А-А 
pai (pth +d)" Е 


4. 


Proof. By Theorem 5 and induction, it is easy to prove (а) and (b). For (с), we follow the hint. If М has 
the spectral resolution M = У)“, AP, log M is defined as 


k А 5 ái " k i k ^ 4 
log M = ores = 25 i -1)P; = lim m (Sa P- у) = dim m(M* - I). 
So log M = limp... m(M** — I) € lim, т(А — I) = log N. a 
5. 


Proof. (from the textbook's solution, pp. 291) Choose A and B as in Exercise 3, that is positive matrices 
whose symmetrized product is not positive. Set 


M=A,N=A+tB, 
t sufficiently small positive number. Clearly, M < N. 
N? = A? + (AB + BA) - B; 


for t small the term #2В is negligible compared with the linear term. Therefore for t small N? is not greater 


than М?. o 
6. 
Proof. For f(z) =az +b- fo 20, we have 

Ы dm(t) 


fG*a2-fG) ase Ac | т==г 


So if we can show limaz—o f; rc exists and is finite, f(z) is analytic by definition. Indeed, if 
Imz > 0, for Az sufficiently small, we have 


1 1 1 2 
< < < . 
z+Az+t|~ e -|Az ` Imz- [Az] ` Imz 
So by Dominated Convergence Theorem, lima. o f° qrati exists and is equal to fj" £}, which 
is finite. To see Im f(z) > 0 for Imz > 0, we note 
* — dm(t) e dm(t) 
Imf(z) -almz-Im | .—— 77 =m ALIAPU 
fe) cete b Rez+t+ilmz [a +), (ertt? mz 


T. 


Proof. Consider the Euclidean space L?(—oo, 1], with the inner product (f,9) := f? f(t)g(t)dt. Choose 


J; = 630-0), j =1,--- ‚т, then the associated Gram matrix is 


1 olretrs)t 1 
Gij = (fs fi) =| Sere aan 
Clearly, (fj)? are linearly independent. So С is positive. п 


8. 
Proof. We apply the change of variable formula as follows 


NR 2т роо 1 
/ ей = Í cP VP dedy = f 40 | етта = (т. = 
—оо R2 0 0 2 


9. 


Proof. The extension is straightforward, just replace the paragraph (on page 161) “If S is a subspace of V, 
then Т = S and dim T = dim S. ... It follows that 


dim S — 1 < dimT 
as asserted." with the following one: Let T= SQV and Т = $n V+, where V+ stands for the compliment. 
of V in U. Then dim T + dim Т = dim S. Since dim T; < dim V+ = n — (n — т) = m, dimT > dim – m. 
The rest of the proof is the same as the proof of Theorem 14 and we can conclude that 


pi(A) -m € p«(B) € p«(A). 


10. 


Proof. For any т, (rz, (N — M — dI)z) = (x, (N — M)z) - d|z? < |N — М||т|? — d|z|? = 0. Similarly, 
(a, (M ANE ara) = (2, (M —N)2) - diz <1M АР —dizf? < о. 0 


11. 


Proof. It’s easy to see the problem can be reduced to the case k = 2. To prove this case, we note if mı < ma 
and np, > Np, we have 


mMaNp, + тїпр, — m2np, — татр, = (ma — mi)(np, — пр) 2 0. 


12. 
Proof. Assume Z is not invertible. Then there exists т 4 0 such that Zz = 0. In particular, this implies 
(x, Zax) = (x, Z*z) = 0. Sum up these two, we get (z, (Z + Z*)z) = 0. Contradictory to the assumption 
that the selfadjoint part of Z is positive. For any т 3 0, there exists y #0 so that 2 = Zy. So 

(т, (Z(27)*)2) = (т, Z 12) (e (Z)*2) = (Zy, y)+(Z* 2, £) = (y, Z*y) +(y, Zu) = (y,(Z+Z")y) > 0. 
This shows the selfadjoint part of Z-! is positive. o 
13. 
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Proof. Exercise Problem 14 has proved the claim for non-zero eigenvalues. Since the dimensions of the spaces 
of generalized eigenvectors of AA* and А* А are both equal to the dimension of the underlying Euclidean 
space, we conclude by Spectral Theorem that their zero eigenvalues must have the same multiplicity. О 


14. 


Proof. Suppose а is a non-zero eigenvalue of АА" and т is an eigenvector of АА" pertaining to а: AA*z = az. 
Applying A* to both sides, we get A*A(A*z) = aA*z. Since a # 0 and x 7 0, A*z # 0 by АА*т = az. 
‘Therefore, a is an eigenvalue of A*A with A*z an eigenvector of A*A pertaining to a. By symmetry, we 
conclude AA* and А* A have the same set of non-zero eigenvalues. 

Fix a non-zero eigenvalue а, and suppose T1, ---, tm is a basis for the space of generalized eigenvectors 


of АА" pertaining to a. Since a # 0, we can claim A*z;, ---, A*z,, are linearly independent. Indeed, 
assume not, then there must exist a1, ---, @m not all equal to 0, such that 7", a; A*z; = 0. This implies 
a(S, oum) = Yuao:AA'; = A(S, а;А*т;) = 0, which further implies т, ---, £m are linearly 


dependent since а # 0. Contradiction. 

This shows the dimension of the space of generalized eigenvectors of AA* pertaining to a is no greater 
than that of the space of generalized eigenvectors of A*A pertaining to a. By symmetry, we conclude the 
spaces of generalized eigenvectors of AA* and А* А pertaining to the same nonzero eigenvalue have the same 
dimension. Combined, we can conclude AA* and A*A have the same non-zero eigenvalues with the same 
(algebraic) multiplicity. a 


Remark 13. The multiplicity referred to in this problem is understood as algebraic multiplicity, which is 
equal to the dimension of the space of generalized eigenvectors. 


15. 

1+bi 3 s Я 
Proof. Let 2 = 0 L4+bi where b could be any real number. Then the eigenvalue of Z, 1 + bi, 
has positive real part. Meanwhile, Z + Z* = ^ Ы has characteristic polynomial p(s) = (2 — s)? – 9 = 
(s — 5)(s + 1). So Z + Z* has eigenvalue 5 and —1, and therefore cannot be positive. o 


16. 

Proof. Suppose A and B are selfadjoint. Then for any x and y, 

(x, (AB BA)*y) = ((AB — ВА)т, у) — (ABz, y) — (BAr, у) = (x,BAy) — (x, ABy) = (x, -(AB — BA)y). 
So (AB — BA)* = -(AB — BA). o 


11 Kinematics and Dynamics 
1. 
Proof. We note 4(M(t)M*(t)) = M(t)M*(t) + M(t)M*(t) = A(t) + A*(t) = 0. So M(t)M*(t) = 


M(0)M*(0) = I. Also, f(t) = det M(t) is continuous function of t and takes values either 1 or -1 by 
the isometry property of M(t). Since f(0) = 1, we have f(t) = 1. By Theorem 1, M(t) is a rotation for 


every t. a 
2. 
Proof. limno MCE2- MO) — limpo EAD = Ae. ie. M(t) = AM(t). Clearly M(0) = I. a 
3. 
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Proof. The reason we need commutativity is that the following equation is required in the calculation of 
derivative: 


1 1 th t 
1 3 1 (Л Абда» _ eff Абда 
a (M(t+h) - M()) x (e e ) 


i efi AG)de [15^ A(s)ds _ A fg At) 


тей A(9ds (o [15^ A(s)ds _ 1) . 
ie. elo AG)ds-- [;*^ A())ds — efo A(s)dsef'*" А(з)4з_ So when this commutativity holds, 


M(t) = Jim (ме +h) - M(t) = M(DA(). 


o 
4. 
Proof. If f = (a,y,z)* satisfies Af = 0, we must have 
ay+bz=0 
-ат +с2 = 0 
—br — cy = 0. 
By discussing various possibilities (a, b, c = 0 or not), we can check f is a multiple of (—c, b, —a)?. n 
5. 
Proof. 
Aya = 
det(sI - A) = де[а А -c] = AQ? +e?) —a(—ad + bc) + b(ac + bA) = A? + A(c? + 7 + a?). 
boc xX 
Solving it gives us the other two eigenvalues. o 
6. 
0 a b 
Proof. Since A= | -a 0 с | is antisymmetric, M(t)M*(t) = ее" = ee = Г. By Exercise 7 of 
-b -c 0 


Chapter 9, all eigenvalues of e^t has the form of е, where a is an eigenvalue of A. Since the eigenvalues 
of A are 0 and tik with k = va? +b? + с? (Exercise 5), the eigenvalues of e^t are 1 and е“, This 
implies det e4t = 1. е. ет = 1, By Theorem 1, M = e“ is a rotation. Let f be given by formula 
(16). From Af = 0 we deduce that ей = f; thus f is the axis of the rotation e^t. The trace of 
e^t is 1+ e + еті = 2coskt + 1. According to formula (4)', the angle of rotation 0 of e^t satisfies 
2cosÓ + 1 = tre^'. This shows that 0 = kt = va? +P +e. o 


T. 


Proof. (AB – BA)* = (AB)* — (BA)* = B* A* — A*B* = (-B)(-4) - (-A(-B) = BA- AB = -(AB – 
BA). o 


8. 
Proof. See the solution in the textbook, page 294. o 
9. 
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Proof. See the solution in the textbook, page 294. a 
10. 


Proof. It suffices to note that the set of 2n functions, {(cos cjt)hj, (sin c;t)h;)7..,, are linearly independent, 
since any two of them are orthogonal when their subscripts are distinct. a 


12 Convexity 
The following results will help us understand some details in the proofs of Theorem 6 and Theorem 10. 


Proposition 6. Let S be an arbitrary subset of X and x an interior point of S. For any real linear function 
1 defined on X, 11 #0, then I(x) is an interior point of Г = ((S) in the topological sense. 


Proof. We can find y € X so that (у) #0. Then for t sufficiently small, I(x) + tl(y) = Ua + ty) € T. So T 
contains an interval which contains l(z), i.e. I(x) is an interior point of Г under the topology of R!. п 


Corollary 1. If К is an open conver set and is a linear function with | # 0, Г = ЦК) is an open interval. 
Proof. Note Г is convex and open in R! in the topological sense. о 


Proposition 7. Let К be а conver set and Ко the set of all interior points оў К. Then Ko is conver and 
open. 


Proof. (Convexity) Vr, y € Ko and a € [0,1]. For any z € X, [ar--(1—a)y]--tz = a(z--tz)--(1-a)(y--tz) € K 
when t is sufficiently small, since x,y are interior points of K and K is convex. 

(Openness) Fix т € Ko, Vy, € X. We need to show for t sufficiently small, z--£y; € Ko. Indeed, Ууз € X, 
we can find a common є > 0, so that whenever (Ё, t2) € [—e,¢] x [—є,є], + Фуу € К and x + toy» € К. 
Fix any t* € [-5,$], by the convexity of К, r+ t*yi + t**yo = $(z + 2t*yi) + (т +2°*ул) € К when 
t** € [-5,5]. This shows z + у € Ko. Since t* is arbitrarily chosen from [—5, 5], we conclude for t 
sufficiently small, т + ё € Ko. That is, x is an interior point of Ko. By the arbitrariness of т, Ko is 


open. о 
dc 
Proof. The verification is straightforward and we omit it. о 
2. 
Proof. These propositions are immediate consequences of the definition. of convexity. n 
3. 


Proof. Fix a point x € {z : (г) < c). For any y € X, f(t) = I(x + ty) = U(x) + tl(y) is a continuous function 
of t, with f(0) = (т) < с. By continuity, f(t) < c for t sufficiently small. So x + ty Є {z : (г) < c) for t 
sufficiently small, ie. т is an interior point. Since т is arbitrarily chosen, we have proved {z : l(z) < c} is 
open. [m] 


4. 


Proof. The convexity of А + В is Theorem 1(b). To see the openness, Vr € A, y € В. For any z € X, 
(к + у) +tz = (т + tz) + у. For t sufficiently small, т + tz € A. So (£ +y) +tz € A + В for t sufficiently 
small. This shows A + B is open. o 


5. 


Proof. That К is a convex set is trivial to see. It's also clear that р(0) = 0. For any т € R” V {0}, when 
e € (0,a), re = 181 satisfies т. > 0 and т/т. € K. So p(x) < El. By letting € — 0, we conclude 
p(x) € [т|/а. If “ <” holds, we can find r > 0 such that r < El and = € K. But r < El implies a < El 


and hence = ¢ К. Contradiction. Combined, we conclude р(х) = El. п 
6. 
Proof. See the textbook’s solution. п 
т. 


Proof. Vz,y Є К, we have р(х) < 1 and p(y) < 1. For any a € [0,1], p(ax+(1—a)y) € р(ат) +р((1-а)у) = 
ap(x) + (1 — a)p(y) < a + (1— а) = 1. This shows К is convex. То see K is open, fix т € К and choose 
any 2 € X. Then p(z + tz) < p(x) + tp(z). So for t sufficiently small such that tp(z) < 1 — р(х), we have 
p(z tz) € p(x) + tp(z) < p(x) + 1— p(z) = 1, ie. r--tz € К. This shows К is open. o 


8. 
Proof. We > 0, there exists z(£) € S, so that 


qs(m +1) = sup(l + m)(z) < (1+ m)(z(e)) +£ € supl(z) + supm(z) + = = qs(m) + qs (l) + £. 
Tes zes тє8 


By the arbitrariness of =, we conclude qs(m + l) < qs(m) + qs(l). o 
9. 


Proof. qs+0(l) = SUPzes,yer l(z + y) = sup;esyerl[l(z) + U(y)] < suPresyerlas(l) + ет()] = as(7) + r0). 
Conversely, Ve > 0, there exists то € S, yo € T, s.t. qs(I) < l(zo) + 5. ar (l) « l(yo) + 5. So as(l) + ar(l) < 
l(zo +9) + € € @з+т(1) +=. Ву the arbitrariness of £, qs(l) + qr(l) <-qs+r (l). Combined, we get 
qs«r(l) = 45(1) + (1). o 


10. 


Proof. qsur(l) = sup,esur!(r) > sup,es!(r) = qs(l). Similarly, gsur(l) > qr(l). Therefore, we have 
qsur(l) > max{qs(l),qr(l)}. For any £ > 0 sufficiently small, we can find т. € SUT, such that qsur(l) < 
Uae) +e. But I(ze) < max{qs(l), ar(1)). So gsur(l) < шах{а5(1), ат()} +=. Let € > 0, we get qsur(t) < 


max(qs(I), qr (1)}.СошЫпей, we can conclude qsur (I) — max(as(l), 4т(1)}.. o 
11. 
Proof. If for any a € (0,1), (ах + (1 — a)y) = al(z) + (1 — a)l(y) < c, by continuity, we have (т) < c and 
Қу) < c. This shows {x : I(x) < c} is a closed convex set. o 
12. 


Proof. Convexity is obvious. For closedness, note f(t) = |tz + (1 — t)y| is a continuous function of t. So if 
f(t) € t for any t € (0,1), f(0) = [у] € 1 and f(1) = [т] € 1. So the unit ball B(0, 1) is closed. Combined, 
we conclude B(0, 1) = {x : || < 1} is a closed convex set. o 


13. 


Proof. Suppose H and K are closed convex sets. Theorem 1(a) says H N K is also convex. Moreover, if for 
any a € (0,1), az + (1 — a)y € H NK, then the closedness of H and K implies a,b € Н and a,b € K, i.e. 
a,b € HNK. So HNK is closed. o 


14. 
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Proof. Proof of Theorem 7: Suppose К has an interior point то. If a linear function l and a real 
number c determine a closed half-space that contains К — то but not y — zo, ie. U(x — zo) < c, Yz € К and 
Қу-то) > c, then I and c--L(zo) determine a closed half-space that contains К but not y, i.e. U(x) < c--l(zo) 
and [(у) > c--l(zo). So without loss of generality, we can assume то = 0. Note the convexity and closedness 
are preserved under translation, so this simplification is all right for this problem's purpose. 

Define gauge function px as in (5). Then we can show px (z) < 1 if and only if z € K. Indeed, if x € K, 
then px (x) < 1 by deifinition. Conversely, if px(x) < 1, then for any = > 0, there exists г(є) < 1+ = so that 
тє) € К. We choose r(e) > 1 and note 5 = a(c) -0+ (1— а(є))-т with a(e) = 1— "S3 As г(є) can be as 
close to 1 as we want when = | 0, a(e) can be as close to 0 as we want. Meanwhile, 0 is an interior point of 
К, so for т large enough, = € К. This shows for a close enough to 1, a - 0 + (1— a)-z € K. Combined, we 
conclude К contains the open segment (a -0-- (1 —a) - :0 « a « 1). By definition of closedness, z € К. 
The rest of the proof is completely analogous to that of Theorem 3, with p(x) « 1 replaced by p(x) < 1. 

1f K has no interior point, we have two possibilities. Case one, y and K are not on the same hyperplane. 
In this case, there exists a linear function / and a real number c, such that I(x) = c(Vr € K) but l(y) 7 c. 
By considering —I if necessary, we can have I(x) = c(Vr € К) and [(у) > c. So the half-space {x : I(x) < c) 
contains K, but not y. Case two, y and K reside on the same hyperplane. Then the dimension of the 
ambient space for y and K can be reduced by 1. Work by induction and note the space is of finite dimension, 
we can finish the proof. 

Proof of Theorem 8: By definition of (16), if x € К, then I(x) < qx(I). Conversely, suppose y is not 
in К, then there exists | € X^ and a real number c such that I(x) < c, Vr € K and [(у) > c. This implies 
L(y) > ак(1). Combined, we conclude x € К if and only if I(x) € qx(l), Vl € X’. 

Remark: From the above solution and the proof of Theorem 3, we can see a useful routine for proving 
results on convex sets: first assume the convex set has an interior point and use the gauge function, which 
often helps to construct the desired linear functionals via Hahn-Banach Theorem. If there exists no interior 
point, reduce the dimension by 1 and work by induction. Such a use of interior points as the criterion for a. 
dichotomy is also present in the proof of Theorem 10 (Carathéodory). o 


15. 


Proof. Denote by 5 the closed convex hull of 5, and define Г = {= : I(x) < qs(l)} where } € X’. Then it 
is easy to see each T; is a closed convex set containing S, so $ €MexTi. For the other direction, suppose 
Piex Ti \ S # 0 and we choose a point т from Cie Ti \ 8. By Theorem 8, there exists lo € X’ such that 
lo(x) > qg(lo) > qs(lo). So x ¢ Гь, contradiction. Combined, we conclude $ = nyexiTi = {x : U(x) < 
qs(l), Vl € X"). 


16. 

Proof. Suppose А1, «++, Am satisfy Xi, =, A € (0,1) and У", № = 1. We need to show Yon € 

К, where K is the convex set to which zi, :::, 2m belong. Indeed, since У", Aim = (А +- 
Am-1i) Ss xv + Аата, it suffices to show Уу-у ete € К. Working by induction, 

we are done. a 
17. 


Proof. Suppose т is an interior point of К. Vy € X, for t sufficiently small, x + ty € К. In particular, we 
can find = > 0 so that z +ey € К and т — ғу € К. Since т can be represented as т = 0+0), we 
conclude т is not an extreme point. o 


18. 


Proof. Let S be a permutation matrix as defined in formula (25). Then clearly S;; > 0. Furthermore, 
DL Siz = Di plij» where j is fixed and is equal to p(io) for some ig. So Dy", Sij = 1. Finally, 
Mia Sig = Whar Sip-1(j)> Where i is fixed and is equal to p^! (jo) for some jo. So 3 77., Sij = 1. Combined, 
we concleude 5 is a doubly stochastic matrix. d 


19. 


Proof. The textbook’s solution demonstrates the case of dimension 3. Counterexamples for higher dimensions 
can be obtained by building permutation matrices upon the case of dimension 3. п 


20. Suppose K is a convex subset of an n-dimension linear space X. We have the following properties. 
(1) If z is an interior point of К in the linear sense, then т is an interior point of К in the topological 
sense. Consequently, being open in the linear sense is the same as being open in the topological sense. 


Proof. Let e1, ---, €, be a basis of X. There exists = > 0во that for any t; € (—e,¢), z--tje; € K,i=1,-+-,n. 
For any y € X which is close enough to т, the norm of y — т can be very small so that if we write y as 
y = za ае, |а| < £. Since for t; € (— +n), 2+ 2, tiei = Уң, 2 (@ + ntie:) € К 
by the convexity of K, we conclude y € K if y is sufficiently close to x. This shows z is an interior point of 
K. o 


(2) If K is closed in the linear sense, it is closed in the topological sense. 


Proof. Suppose (тк), C K and zy — т in the topological sense, we need to show z € К. We work by 
induction. The case n = 1 is trivial, because т is necessarily an endpoint of a segment contained in K. 
Assume the property is true any n < N. For n = N + 1, we have two cases to consider. Case one, K has 
no interior points. Then as argued in the proof of Theorem 10, K is contained in a subspace of X with 
dimension less than n. By induction, K is closed in the topological sense and hence x € K. Case two, K has 
at least one interior point то. In this case, all the points on the open segment (ro, z) must be in K. Indeed, 
assume not, then there exists an т* € (20,1) such that the open segment (r9, z*) C K, but (2*, 2] N K = 0. 
Since те is an interior point of K, we can find n linearly independent vectors e}, ---, e, so that x +e; € K, 
i = 1,::-,п. For any ту sufficiently close to x, the cone with тк as the vertex and х0 + €1,::* ,% + ên 
as the base necessarily intersects with (z*,z]. So such an z* € (ro, z) with (r*,z] N К = 0 does not exist. 
Therefore (20,2) C К and by definition of being closed in the linear sense, we conclude x € К. [m] 


(3) If K is bounded in the linear sense, it is bounded in the topological sense. 


Proof. Assume K is not bounded in the topological sense, then we can find a sequence (7.);*., such that 
121 = оо. We shall show K is not bounded in the linear sense. Indeed, if the dimension n = 1, this is 
clearly true. Assume the claim is true for any n < N. For n = N +1, we have two cases to consider. 
Case one, K has no interior points. Then as argued in the proof of Theorem 10, K is contained in a 
subspace of X with dimension less than n. By induction, K is not bounded in the linear sense. Case two, 
К has at least one interior point то. Denote by yx the intersection of the segment [ro, xx] with the sphere 
S(ro,1) = {z : |z — то] = 1). For k large enough, уь always exists. Since a sphere in finite-dimensional 
space is compact, we can assume without loss of generality that yy — у € S(zo, 1). Then by an argument 
similar to that of part (2) (the argument based on cone), the ray starting with y and going through y is 
contained in K. So K is not bounded in the linear sense. o 


13 The Duality Theorem 
1. 
Proof. Let Y = {y : y = Di") pjyj-P; > 0}. If y, y' € Y, then 


ty + (1—t)y = у ltp; + (1 — 0р] € Y. 


i=1 


So Y is a convex set. o 
2. 
Proof. £x — £z = €(z — 2) > 0. o 
3. 
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Proof. In the proof of Theorem 3, we already showed that there is an admissible p* for which yp* > s 
(formula (21)). Since S > yp* > s > S by formula (16) and (20), the sup in Theorem 3 is obtained at 
p*, hence a maximum. To see the inf in Theorem 3 is a minimum, note under the condition that there are 
admissible p and £, Theorem 3 can be written as 


sup{yp: y > Yp, p 2 0} = inf(£y: < £Y, € > 0} # оо. 
This is equivalent to 
inf {(—7)p : (-y) € (—Y)p,p > 0} = sup{€(—y) : (—7) 2 £(-Y),£ > 0}. 
By previous argument for S = sup, yp, we can find £* such that £* > 0, (—7) > €*(-Y) and €*(-y) = 


sup{€(—y) : (-7) 2 €(-Y),£ > 0), i.e. & > 0, у € &'Ү, and &*у = inf {Ey : y < EY, € > 0). That is, the inf 
in Theorem 3 is obtained at £*, hence a minimum. п 


14 Normed Linear Spaces 


1. 
Proof. Trivial and proof is omitted. о 
2. 
Proof. |x — z| = 1(2– у) + (y - 2)1 < le 01+ ly — zl- п 
3. 


Proof. (i) |т| > 0, and |z|1 = 0 if and only if each a; = 0, ie. z = 0. 
Gi) Je + yh = 012 + wl € 3215] +X ysl = |т + lyh- 
(iii) [har], = X |kz;| = 221123 = Iklil- d 


4. 


Proof. f(x) = — nz is a strictly convex function on (0,00). So for any a,b > 0 with a 7 b, we have 
f(0a + (1 — 6)b) € 0f(a) + (1 —0)/(0), YO € [0,1], 


where “=” holds if and only if ба + (1 — 0)b = a or b. That is, one of the following three cases occurs: 1) 
6=0;2)0=1;3)a=b. 
We note inequality f(0a + (1 — 0)b) € 0f (a) + (1 — 6)f (b) is equivalent to 001-9 < ба + (1 — 0)b, and by 
zi Q 
letting a; = oe and b; = wp, we have (0 = 3) 


тд 
ШЕ 


l|nf , 1 |ail? 


plei а lel 


Taking summation gives У), |z;y;| < |zlplylg- 
We consider when У, |z;y;| = |r|p|y|;- Since p,q are real positive numbers and since 1 + 1 = 1, we must 
have p,q € (0,1). So among the three cases aforementioned, “=” holds in У), |2;y;| < ||, |y|, if and only if 
for each i, ар = шр, that is, (|2,|?,--+ ,[2, |?) are proportional to (|з [*, - -- , |, |"). 
For У), тии = У [z;yi| to hold, we need туу; = [riy;| for each i. This is the same as sign(r, 
for each i. In summary, we conclude zy < |z|p|y|q and the “=” holds if and only if (|z: |P,- 
(lu 5. -+ ,[ya]2) are proportional to each other and sign(z;) = sign(y;) (i = 1,--- ,n). 


5. 
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Proof. Given x, we note 


6. 


Proof. Every linear subspace of a finite-dimensional normed linear space is again a finite-dimensional normed 
linear space. So the problem is reduced to proving any finite-dimensional normed space is closed. Fix a basis 
€1,77 „ёп, We introduce the following norm: if т = У ajej, |x| := (37; a2) /?. Then the original norm | - | is 
equivalent to | - |. So (2,)72, is а Cauchy sequence under | - | if and only if {(ax1,-++ ,@kn)} 72, is a Cauchy 


sequence in C^ or В". Here ть = "_,a,j¢;. Since С" and В" are complete, we conclude there exists 


(bi, ++- ,bn) ЄС" or R”, so that zy > т = Убе in |- | and hence in |. |. a 
т. 


Proof. If (20), C Y is such that |x — | > d := infyey |x — y|, then for k sufficiently large, |26| < 
[ак — z| + |z| € 2d + |z|. Note that Y = span{y;,--- ,yn} is a finite dimensional space, by Theorem 3 (ii), 
(zk)? has a subsequence which converges to a point yo € Y. Then infyey |x — y| is obtained at yo. o 
8. 

Proof. (i) Positivity: ||! = 0 implies ёт = 0, Vx with |z| = 1. So for any y with y Z 0, £y = |yl£(y/lyl) = 0, 
ie. € = 0. So ||’ = 0 implies £ = 0, which is equivalent to £ # 0 implies |£|' > 0. |0| = 0 is obvious. 
Subadditivity: | + £2| = Z jaj=1 (£1 + 2)@ < sup; &2 + вар „у фт = [61| + [2]. 

(iii) Homogeneity: |kE| = вар kEa = |k|supjz|=1 & = |К||&|. o 


9. (i) 


Proof. By formula (47) and (48), it suffices to prove the equality for positive rational r. Suppose r = i with 
p,q € Z+. By formula (49) and by induction, we have 


———— 
(x,y) +--- + (т,у)= (nz, y). 


"Therefore 4 
р(тт, y) = (przy) = (дї, y= qlz, y) ie. (rz, y) = po?» = г(т„у). 


(i) 
Proof. For any given k, we can find a sequence of rational numbers (т,)22.; such that т, — k as n — оо. 


Then k(x, y) = lim, oo rs (y) = lim, ls (rz, y) = (lim, ss тат, y) = (Кт, y), where the third “=” uses 
the fact that (-,y) defines a continuous linear functional on X. a 
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15 Linear Mappings Between Normed Linear Spaces 

A. 

Proof. By Lemma 1, [Tr —Tz| = |Т(т„—т)| € с|т„—т|. So if limno £n = z, then lim, Tr, = Tz. O 
2. 


Proof. Suppose |Т„ — T| does not tend to zero. Then there exists £ > 0 and a sequence (x,)°., such that 
len| = 1 and (Т„—Т)ть| > £. By Theorem 3(ii), we can without loss of generality assume (х); converges 
to some point z*. Then 

I7 T)ss|- (7,7 T)z*| € (75 -T)z.- (75 -T)2*| € |Т(т„—т*)|+1Т„(т„—т°*)| € |Т|ж„—х*|+|Т„[|„—т°*|. 
For n sufficiently large, |(Т„ — Т)г„| — (T — T)z*| will be greater than ¢/2, while |T]|z;, — т*| + |Т„]|т„ —*| 
will be as small as we want, provided that we can prove (|Т„ |), is bounded. Indeed, this is the principle 
of uniform boundedness (see, for example, Lax [6], Chapter 10, Theorem 3). Thus we have arrived at a 
contradiction which shows our assumption is wrong. a 


Remark 14. Can we find an elementary proof without using the principle of uniform boundedness in func- 
tional analysis, especially since we are working with finite dimensional space? 


3. 


Then we note 872, R* = No" -EZ АК = I and (Zo R*)S = YT RE — YER = 1. So S 
is invertible and $7! = Yr, o 


4. 


Proof. Assume all the conditions in Theorem 5. Define R = —T-!(S — T), then |R| < |T>+\\S—T| < 1. So 
by Theorem 6, I — R = T-!S is invertible, hence 5 = T o T-!5 is invertible: о 


5. 


Proof. If for some т, |R™| < 1, we define U = Efco R'm = I + R"U. U is well-defined, and the following 
linear map is also well-defined: V = U + RU -- --- + R"-!U. Then SV =U + RU +--+ R7-!U — (RU + 
R?U +--+ RU) =U — R™U = I. This shows S is invertible. [m] 


6. 


Proof. For any т € R", |Tz|;; = шах; | 375. tijzj| < max: (®©; |tijl)|eloc., So [T] = Sup, zo ES 5 
max; 55, |fij|. For the other direction, suppose 57; |tioj|= шах; 27; |tij| and we choose 


Proof. First of all, 72.9 R* is well-defined, since by |R] < 1, (Es в), is а Cauchy sequence in X". 


= (sign(tig1), --- sSign(tign))”» 


then |a*|oo = 1 and Tz* = (30, tijs}, +- У Ноја, У, 25). So |Тт* loo 2 У tii] = maxi 27; |631. 
This implies |T| > max; 37; Teil. Combined: we ЕА ЇЙ = шах 57, tij]. o 
7. 
Proof. For any т = (x1,-++ ,&n)', we have 
[Тт = Xs < Y unl У |н: 
i= i=l j=1 
So |T] = sup, a < 35, ltil- o 


Proof. The proof is very similar to the content on page 97, Chapter 7, the material up to Theorem 18. So 
we omit the solution. o 


16 Positive Matrices 
1. 


Proof. Let 2* = (1,:-- 1)" and А = maxi<i<n 375, Pij, then Pz* < Ат". So Р) # Ü and t*(P) = (0 < 
A < À* : À € t(P)} is a bounded, nonempty set. We show further t*(P) is closed. Suppose (А); C t*(P) 
converges to a point А. Denote by 2" the nonnegative and nonzero vector such that Pz" < A727. Without 
loss of generality, we assume У", 27" = 1. Then (2")9°_; is bounded and we can assume 2” — т for some 
r0 with У z; = 1. Passing to ‘limit gives us Px < Ат. Clearly 0 < A € А". So А € t*(P). This shows 
t*(P) is compact and t(P) has a minimum А. 

Denote by z the nonzero and nonnegative vector such that Рт < Az. We show we actually have 
Pz = Xz. Assume not, there must exist some k € {1,--- ,n) such that 575 ., PijZij € Ат; for i 7 k and 
Уа DkjZj < АсАть. Consider the vector 2 = Z — гер, where £ > 0 and ex has the k-th component equal to 
1 with all the other components zero. Then in the inequality PZ < Ат, each component of LHS is decreased 
when 2 is replaced by 2, while only the k-th component of RHS is decreased by an amount of Ae. So for 
є small enough, Р? < AZ, and we can find a X < X such that Р? < А. Note À > 0 (otherwise z — 0, a 
contradiction), so we can also let X > 0. This contradicts with А = minje;(p) A. We have shown A»0isan 
eigenvalue of P which has a nonzero, nonnegative eigenvalue. By Theorem 1(iv), А = A(P). o 


2. 


Proof. P™ has a dominant positive eigenvalue А. By Spectral Mapping Theorem, there is an eigenvalue А 
of P, such that А" = Ag. Suppose A is real, then we can further assume А > 0 by replacing А with —A if 
necessary. Then for any other eigenvalue А of P, Spectral Mapping Theorem implies (A")"" is an eigenvalue 
of P". So |(M)"| < Ay = А", ie. |X] <А. 
To show we can take A as real, denote by т the eigenvector of P™ associated with Ag. Then 
Р"т = №. 


Let Р act on this relation: 
Р" = pn(Pz) = у Ий 


This shows Pz is too an eigenvector of P™ with eigenvalue Ay. By Theorem 1(іу), Px = cx for some positive 
number c. Repeated application of P shows that Р""т = с". Therefore c" = Ag. Let А = c. ü 


17 How to Solve Systems of Linear Equations 
The three-term recursion formulae 
En41 = (SnA + раГ)тһ + qnzn-1 — Snb 


is introduced by Rutishauser et al. [1]. See Papadrakakis [9] for a survey on a family of vector iterative 
methods with three-term recursion formulae and Golub and van Loan [2] for a gentle introduction to the 
Chebyshev semi-iterative method (section §10.1.5). 


1. 
Proof. I = AA". Зо 1 = |I] =|AA™| < |AlA7!| = (A). a 
2. 


Proof. If we use the method of Section 1, we need to solve for NV the following inequality: 2(-2)NF(z)« 
10-5. Plug in numbers, we have N > 757. If we use the method of Section 2, we need to solve for N the 
inequality 2(1 + ES leo] < 107—3. Plug in numbers, we have N > 42. The numbers of steps needed in 
respective methods differ a great deal. o 


3. 
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Proof. We first summarize the algorithm. We need to solve the system of linear equations Ax = b, where b 
is a given vector and A is an invertible matrix. We start with an initial guess zo of the solution and define 
то = Ато — b. TO BE CONTINUED ... п 


4. We solve the following problem from the first edition of the textbook: Use the computer program їп 
Exercise 3 to solve the system of equations 


с some nonnegative constant. Vary c between 0 and 1, and the order К of the system between 5 and 20. 


Proof. TO BE CONTINUED ... u 


18 How to Calculate the Eigenvalues of Self- Adjoint Matrices 
TO BE CONTINUED ... 


19 Appendix 


19.1 Special Determinants 
1. 


Proof. The system of equations p(a;) = p; (i = 1,--- ,n) can be written as 


iue, -- a Ty m 

1 a) а | |= | || p 

l as c ад) Vw Pn 
Since a), ---, a, are distinct, by the formula for the determinant of Vandermonde matrix, the determinant 
of the matrix is equal to [T;..,(a; — a;). o 


2. 


2 
Proof. Denote the matrix by A. We claim det A = do Indeed, by subtracting column 1 from each 
of the other columns, we have 


1 1 1 1 
Tła? Itaia?  lkaias IFaran 
1 1 1 PES 1 
1+азат T+a3 1+азаз Т+азаһ 
detA = det} Сү zy E us 
Tram Tra Tfaa Traian 
Tana: TFanag 1+аһаз Tea 
1 а1(аз—аз) a1(25—21) ES a1 (an —21) 
Tra] (ї+а?)(1+а1аз) (1+аї)(1:+ааз) xad) (1+алал) 
1 аз(а2—ал) аз(аз ал АЕ аз(а„—а1) 
T¥aga; (О+азаа) 0+0)  (1+аза1)(1+азаз) @+азаз)(1+аза„] 
= det], ач(аз—оз) ax(ag—a1) 2. ааһа) 
Ifaa — (1 aiai)(TFaia2) (ғаға) (1+ааз) П+аа) П+ааһ) 
1 as (22—a1) an (a3—21) Mi an(an—a1) 
1+аһа  (1+anai)(14ana2) (1+anai)(1+ana3) (1+ana1)(1+a3) 
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By extracting the common factor 
column, we have 


(i = 1,--- ,n) from each row and (a; — a1) (j 


we - ,n) from each 


1 а а oes а 


lcraia2 lcraiag l+aran 
И ЗЫ КНЫН a 
Mala; а) Ita? 1Ғағаз TFazan 
detA = 222299 “дыр o an p 
IL, Q + aia) 1 qma Tras 77 Тїш 
1 Theva Tforas 5 тат 
Subtracting row 1 from each of the other rows, we get 
Ir (a ap П+а:аз)(1+а2) Пғагаз)(1+азаз) `` {Т+азаһ)(1+аза„) 
det = T де 0 ical РЕҢ ©. mag 
Th, Faa) Сала ааз)  (Faras)(1 ғагаз) Сатан) (1+атал) 
0 пра аа) Пааа) 7 (езй) 


By the Laplace expansion and extracting the common factor (a; — ау) from row 2 through n and ће common 
factor тиу from column 2 through п, we get 


1 1 1 
Tray Tazas `` Т+азан 
m aa) ^ б sis А 
det A= Пу — a1) det | = b.e qd 
А " А Trai — lraias ТҒаан 
Tina (1 + aia) [2 + 125) Be: xir eile or 
II 1 oe 1 
TFanag Т+алаз faf 
By induction, we can prove our claim. o 


19.2 The Pfaffian 
1. 
Proof. By the Laplace expansion and Exercise 16 of Chapter 5, we have 
Е 0 , is a b c d wb Xe a bc 
dé| 5 a o f adet|-d 0 f|—bdet}0 d e}+cdet}0 d e 
-e -f 0 -e -f 0 -d 0 f 
= a(—bef + cdf + af?) — b(—be? + cde + aef) + c(adf — bde + cd?) 
—2abe f + 2acdf — 2bcde + a? f? + We? + cd? 
= (af —be- cd}. 


Li 


19.3 Symplectic Matrices 
1: 


-а 


b gi dh -iE d 


Proof. We work by induction. For n — 1, A has the form of [ 0 | Since det A # 0, a 7 0. We note 
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Now assume the claim is true for 1, --- , п, we show it also holds for n + 1. Indeed, we write A into the form 
0 a * 
-a 0 * |, 
* * A 
where Ау is a 2n x 2n anti-self-adjoint matrix. Then 
t 0 Oix2n 0 a * 1 0 01x2n. 0 1 * 
0 io О | |-а 0 * 0 l О | = |-1 0 «|. 
Озахі Oznxı nxan] |* * А хі Üonxi вхэп ж * Ay 


Recall that multiplying an elementary matrix from the left is equivalent to an elementary row manipulation, 
while multiplying an elementary matrix from the right is equivalent to an elementary column manipulation. 


A being anti-self-adjoint implies A; = —Aj;, so we can find a sequence of elementary matrices U; , Uo, - +- , Uy 
such that 
0 1 x о1о 
Uj--UXAh|-1 0 *[|UTUT--.Ug-|-1 0 0}. 
жож AL 0 0 A 


By assumption, А = ЁЛЕ for some real matrix F, with det F, 0 and Л = [ des "su . Therefore 
—Inxn Onxn 
(U := Ux: +: U3U1) 


01 + 0 10 

Тәк? 2] ар? 0 
0 о-о „| Sm- [-10 0]. 
Е Bn кк Aj eer) оол 


Define 
01 0 0 
| Uo BD En 
P= Ж.И. 
00 Lx. 0 
and i 
1 0 Oxm]lY- 
I 0 
F= ps Ба 0 1 01x А 
5 O2xi nxi  Donx2n 
Then 
dua d 1 0 Oix2n 0 а * 
FOUA(F)? = [Ке PAL 0 1 Ох | |-а 0 * 
x Üonxi Üonxi Tonxan} | * * А 
1 0 0ix2n 
| 0 io Ом 
Озахі O2nx1  Danx2n 
= [ 0 feriis]. 
—I(n+41)x(n+1) 0 
By induction, we have proved the claim. о 
2: 


Proof. For any given x and y, define f(t) = (S(t)z, JS(t)y). Then we have 


SIO = (ESO ISO) + (9102, J 5.509) 

= (CWS()x, J5(0)9) + (Sx, ICOS) 

= (JL(S(t)x, 78(0) + (Sz. PLOSO) 
(LSx, ITISA) - (800%, LOSH) 


(S(t)z, L(t)S(t)y) — (Sz, LESY) 


0. 
So f(t) = f(0) = (S(0)z, JS(0)y) = (=, Jy). Since x and y are arbitrary, we conclude S(t) is a family of 
symplectic matrices. o 
4. 
Proof. We note 


2n Qv, du. 

de [LU | ay du ө 
X de “it 
Ни) can be seen as a function of v: к(ь) ё = H(u(v)). So 


ок | уг" 4 p" T P =- (&) © 


in. Be рш. Duan: 


Since дь/ди is symplectic, by Theorem 2, ðu/ðv and (ðv/ðu)T are also symplectic. So using formula (4) 


gives us a 5 4 
dv _ дь (dv\" (du ди 
= э (дь) (£) n, - (5) H, 9 JK,. 


19.4 Tensor Product 
19.5 Lattices 
19.6 Fast Matrix Multiplication 


There are no exercise problems for this section. For examples of implementation of Strassen’s algorithm, we 
refer to Huss-Lederman et al. [7] and references therein. 


19.7 Gershgorin’s Theorem 
А 


Proof. Using the notation of Gershgorin Circle Theorem, let B(t) = D + tF, t € [0,1]. The eigenvalues of 
B(t) are continuous functions of t (Theorem 6 of Chapter 9). For t = 0, the eigenvalues of B(0) are the 
diagonal entries of A. As t goes from 0 to 1, the radius of Gershgorin circles corresponding to B(t) become 
bigger while the centers remain the same. So we can find for each d; a continuous path y(t) such that 
^&(0) = d; and y(t) is an eigenvalue of B(t) (0 € t € 1, i = L.---,n). Moreover, by Gershgorin Circle 
Theorem, each path ^;(f) (0 < t < 1) is contained in disc C; = {z : |x — d;| € | fil}. If for some i; and iz 
with i: # io, 71, (0) falls into C;,, then it’s necessary that Су, Ci, 7 0. This implies that for any Gershgorin 
disc that is disjoint from all the other Gershgorin discs, there is one and only one eigenvalue of A falls within 
it. 


Remark 15. There’s a strengthened version of Gershgorin Circle Theorem that can be found at Wikipedia 
(http://en.wikipedia.org/wiki/Gershgorin.circle theorem). The above exercise problem’s solution is an adap- 
tion of the proof therein. The claim: If the union of k Gershgorin discs is disjoint from the union of the 
other (n — К) Gershgorin discs, then the former union contains exactly k and the latter (n — К) eigenvalues 
of A. 


o 


19.8 The Multiplicity of Eigenvalues 
19.9 The Fast Fourier Transform 


There are no exercise problems for this section. 


19.10 The Spectral Radius 


Remark 16. In the tertbook (pp. 340), when the author applied Cauchy integral theorem to get formula 
(20): Ј.-, R(z)z/dz = 2ni A), he used the version of Cauchy integral theorem for the outside region of a 
simple closed curve (see, for example, Gong and Gong [4], Chapter 2, Exercise 8). 

1. 

Proof. If T is an upper triangular matrix with diagonal entries a1, ---, an, then its characteristic polynomial 
рт) = det(AI — T) = П" (Аа). So 


n 
Xo is an eigenvalue of Т & det(AoI — T) = 0 € По» — aj) = 0 & Xo is equal to one of ai, ++- , an. 


i=l 


2. 


Proof. Let D = diag{a,,--- ,a,}. Then De; = diag{0,--- ,0,a;,0,--- ,0}. So |De;| This shows 
|D| 2 maxi<i<n lai]. For any x € С", От = diag{a,r1,--- ,ant,}. So |Dz| = Vila? < 
ü 


maxi <i<a |ai| [т]. So |D] < maxi<i<n |a;|. Combined, we conclude |D] = maxi<i<n lail. 


3. 


Proof. By examining the proof for Euclidean space, we see inner product is not really used. All that has 
been exploited is just norm. So the proof for any finite-dimensional normed linear space is entirely identical 
to that of finite-dimensional Euclidean space. [m] 


4. 


Proof. It suffices to note that a sequence (A,)5*., converges to A in matrix norm iff each ((A,)ij), 
converges to Aj; (Exercise 16 of Chapter 7). 


5. 


n: 


Proof. By formula (16) of Chapter 5: D(a1,-++ ,a4) = Ð a (p)ap,1 `` ap,5; we conclude the determinant of 
any analytic matrix (i.e. matrix-valued analytic function) is analytic. By Cramer's rule and det A(z) Z 0 in 
С, we conclude A7 (2) is also analytic in С. o 


6. 


Proof. By Exercise 4, Cauchy integral theorem for matrix-valued functions is reduced to Cauchy integral 
theorem for each entry of an analytic matrix. o 


19.11 The Lorentz Group 

19.12 Compactness of the Unit Ball 

1. @ 

Proof. We use the notation of Theorem 3. For simplicity, we assume G is convex so that for any z,y € С, 

the segment {z : 2 = (1 — t)z + ty} C С. Then by Mean Value Theorem, there exists c € (0,1) such that 
А) — FY) = УХ — e)z + ey) - (y — x)| < dmlz — yl, 


where d is the dimensional of the Euclidean space in which G resides. This shows the elements of D are 
equi-continuous in G. o 
(ii) 
Proof. From (i), we know each element of D is uniformly continuous. So they can be extended to G, the 
closure of G. Then Theorem 3 is the result of the following version of Arzela-Ascoli Theorem (see, for 
example, Yosida [12]): Let S be a compact metric space, and C(S) the Banach space of (real- or) complez- 
valued continuous functions z(s) normed by |z| = ѕир,сз |2(8)1. Then a sequence (z,(s)) C O(S) is 
relatively compact in C(S) if the following two conditions are statisfied: (a) (2, (s)) is uniformly bounded; 
(b) (2, (5)) is equi-continuous. a 


19.13 A Characterization of Commutators 


There are no exercise problems for this section. 


19.14 Liapunov's Theorem 
1. 


Proof. 'This is basically about how to extend the Riemann integral to Banach space valued functions. The 
theory is essential the same as the scalar case — just replace the Euclidean norm with an arbitrary norm. So 


we omit the details. o 
2. 

Proof. It suffices to note Aj — A in matrix norm if and only if each entry of A, converges to the corre- 
sponding entry of A (see Exercise 7 and formula (51) of Chapter 7). o 
3. 

Proof. We note for T’ >T, by Definition 2 

ro pe 
I ete ttl < [ |e" 4 е: || at. 
т т 


By Lemma 4, limp fT eW" 
exists. 


е" |а = 0. So by Cauchy's criterion, we conclude the integral (12) 
a 


ат 


19.15 The Jordan Canonical Form 


There are no exercise problems for this section. In the below, we give a brief summary of how to compute 
the Jordan canonical form from the standpoint of module theory. 

Oftentimes, we need to compute the Jordan canonical form of a given square matrix A, e.g. in solving 
system of linear ordinary differential equations. The method given in the textbook, although conceptually 
appealing, is still complicated in practice. We hope to find an easier method. 

Qiu [11] (Reading Materials 3, 5, 7) explained a way to compute the Jordan canonical form of A through 
the elementary divisors of the \-matrix A(A) := АГ — A. This is essentially the factorization theorem of a 
finitely generated module over the principal ideal domain K[A], where K[A] is the polynomial ring over the 
field К. A systematic exposition of this theory can be found in Nie and Ding [8] (Chapter 5 and 6) and 
Gong [3]. 

In the following, we will summarize the relevant results for future reference. The exposition combines 
materials from Gong [3] and Qiu [11]: the theoretical aspect is taken from Gong [3], and the computational 
aspect is taken from Qiu [11]. 


19.15.1 Introduction 


Before we dive into abstract theory, we explain the prototype that motivates the theory. Let V be a finite 
dimensional linear vector space over the complex field C. Let A be a linear operator on V. We want to 
choose a suitable set of basis for V such that the matrix representation of A has a nice form, e.g. diagonal, 
block diagonal, etc. 

The key observation is that the matrix of A has a block diagonal form diag{D,, D», - -- , Dy} if and only 
if V can be decomposed into the direct sum of subspaces invariant under A: 


v= -v 


Hamilton-Caley Theorem says P(A) is the zero linear transformation on V (i.e. P(A)v = 0, Vv € V), where 
P(A) is the characteristic polynomial |AJ — A|. In other words, V is the kernel of P(A). This immediately 
reduces the decomposition of V to the factorization of P(A) into irreducible elements in the polynomial ring 
КДА]. Note K[A] is a principal ideal domain, that is, something behaves like the ring of integers. Thus, we 
will have at our disposal the unique factorization theorem of a principal ideal domain into prime elements. 

Extending those concrete objects to an abstract setting, we come to the standpoint of representation 
theory: linear vector spaces equipped with linear transformations are modules over principal ideal domains. 
Therefore, the (concrete) decomposition of V into direct sum of subspaces invariant under A is generalized 
to the decomposition of a finitely generated module over a principal ideal domain into the direct sum of 
primary submodules. 

With all these specific objects in mind (linear space, polynomial ring, characteristic polynomial, integers, 
prime numbers, etc.), we are ready to understand the module theory’s standpoint. 


19.15.2 Decomposition of a finitely generated module over a principal ideal domain 


For the definitions of ring, integral domain and principal ideal domain, we refer to Gong [3], Chapter 1. 
Roughly speaking, they are abstract generalizations of the integer ring Z and the polynomial ring K[A] over 
a field K. Here, the nontrivial conceptual leap is the move from integral domain to principal ideal domain. 
Integer ring Z and the polynomial ring КА] are principal ideal domains because of Eulidean algorithm. The 
Unique Factorization Theorem for Z and K[A] can then be generalized to principal ideal domain. That's 
sufficient for our purpose of decomposing a linear vector space. 

The connection between Unique Factorization Theorem of K[A] and the decomposition of V is made by 
the concept of module. More precisely, define the action of f(A) € K[A] on V by 


fv = f(A)v, Vv € V. 


If V = ker(f(A)) and f(A) has the factorization into non-associated prime elements f(A) = IT^ „[0:(А)]®, 
we have V = QD? , ker(p**). 


For a general module, various technical conditions are needed to produce a clean-cut form of Unique 
Factorization Theorem. The relevant results are listed below. For all the jargons, results, and proofs, see 
Gong [3], Chapter 4. 


Theorem 1. (Cyclic decomposition theorem of a finitely generated module over a principal 
ideal domain — in terms of elementary divisors) Suppose М is a non-zero finitely generated module 
over a principal ideal domain R. Then 


M = Mir Myre: 


where Migr is the collection of torsion elements in М and Ме is a free module, whose rank is uniquely 
determined by M. 

ann(M) = {r € R: rM = {0}} is an ideal of R and hence is generated by some element р^ ---pt» € R. 
Here p;’s are prime elements which are not associated to each other. Then 


Mtor = Mj CD: D Mpa: 


where My, = (v : рўе = 0} is a primary module with order p;*, i = 1,--- n. 
Each Mp, can be further decomposed into the sum of cyclic submodules 


Mp, = Ci BO: Bax, 
where Ci j has order р; (ў = 1,--- ,ki) and 


ei = ein 2 єз >. 26g 2l i=1,---,n. 


fid a 


D; "'s are called elementary divisors and are uniquely determined by M up to the equivalence of associa- 
tivity. In summary, we have 


M = (a QD O r) D Qoa Qoq us. 


Note if S and T are cyclic submodules of M with ann(S) = (a) and ann(T) = (b), and gcd(a,b) = 1, 
then SNT = (0) and S QD T is also a cyclic submodule with ann(S QD T) = (ab). With this observation, 
let Di = Cii @--- @ Сл, then Dy is a cyclic submodule with order q; = Пр". Define D», ++, Din 
similarly, where m — max;(k;). More precisely, we can arrange the elementary divisors into the following 
array 


qux 
po" ce prt 
pie к” 
ifj < E б 
where eij = h Visa Define ф = П", j = 1,---,т. We also arrange the invariant 
subspaces into the following array 
Cu c Cim 
Сы c Com 
Сы c Cum 
Cig, ifj < ki m Р 
where Су = { (0), otherwise’ Ре Dj = Biba Cig j= Aye m. 


Theorem 2. (Cyclic decomposition theorem of a finitely generated module over a principal 
ideal domain — in terms of invariant factors) Suppose M is a finitely generated module over a principal 


ideal domain R, then 
M = D: Q- Q Dm Q мд. 
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where Mfree is а free submodule of M, and D; is a cyclic submodule of М with order qj, j = 1,--- m. 
Moreover 


m|Gn—13 Im—1|Gn—2>*** ›Ф]Ч- 


qi, i = 1, m are called invariant factors of M, which are uniquely determined by M up to the equivalence 
of associativity. 


19.15.3 Decomposition of a finite dimensional linear vector space under a linear operator 


Now we come back to our original problem: given a linear vector space V over a field K, for a given linear 
operator A on V, find a suitable set of basis for V under which the matrix representation of A has a nice 
form. 

In view of the module theory presented in the previous section, we note the polynomial ring КА] is а 
principal ideal domain and V is a finitely generated torsion module over K[A] with the action defined by 
p(A)a = p(A)a, Vp(4) € K[A] and a € А. ann(V) = {р(А) € KIA] : p(A)V = {0}} is an ideal of K[A], 
so there exists a unique m(A) € K[A] such that the coefficient of the term with highest degree is 1 and 
ann(V) = (m(A)). m(A) is called the minimal polynomial of A. 


Theorem 3. Suppose V is a finite dimensional linear vector space and A is a linear operator on V. Suppose 
the minimal polynomial m(A) of A has the decomposition into prime elements 


m(A) = рт (A) pe (A), 
where p;(), i = 1,--- ,n are non-associated monic order polynomial. Then V can be decomposed into direct 


sum 
V = Va O O Von 


where Vp, = (v € V : pi (A)u = 0} and the minimal polynomial of A|v,, is p; (X); i = 1,- 
Each Vp, (i — 1,--- ,n) can be further decomposed into the direct sum of cyclic subspace 


ei = 64,1 26i22- > ещ > 1. 
The elementary divisors p; ^ (4) of V are uniquely determined by A. In summary, we have 


V = (ra) QD Pere) PB Quo Ә-Ә). 


If дер“ (А) = dij, then 


Bij = (vig, Avi, -- , A7 уу 
is а basis of (vij). The matrix of A relative to the basis В = (Вуд, ,Bn,k,) із a block diagonal matriz 
Cip: ^ (А) 
Cipi ^ QJ] 
[A]s — 
Ср“ QJ] 


сг (А) 
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where 


оо - 0 -% 

10- 0 -a 
C[p(z)):- |0 1 

Poi 7e 0 -ы—2 

0 0 =- 1 -a-i 


for p(x) = ao - ayz a4 4277! +2". The block diagonal matriz [А]ӊ is called the rational canonical 
form of A. 


As a by-product, by using the rational canonical form of A, we can see the characteristic polynomial 
ГАГ — A| is equal to the product of elementary divisors [], ; p^ (A). 
When K is algebraically closed, the minimal polynomial m(A) of A can be decomposed into product of 
linear factors. This allows us to get a simpler form of the matrix representation of A. 
Theorem 4. If the minimal polynomial m(X) of A can be split on К, i.e. 
m(z) = (z А)" --(z А)", 


then V can be decomposed into 
V = (ma) PB Qi) QD QI 0 PB: Qi. 
where (vi j) is a cyclic subspace of V and the minimal polynomial of A|,, ‚у is (т — Ai) with 
е = ел 2622-26 21 
These elementary divisors are uniquely determined by A. Let 


бы = (vig (A — А), (A A)T v). 


Then Gi,j is a basis for (vij) and the matrix representation of A under the basis G = (G1,1,- +- ,Gn,kn ) is the 
Jordan canonical form 
900,611) 
(еы) 
[А]в = - , 
9n. 641) 
9. 6,4... 


where 9(Xi,e:,j) is the eij x eij matriz 


м 1 0 0 
0X 1 0 
9056)2|0 0 X 
S ATE e е NE 
0 0 + ^ 


So far so good, but a fundamental question remains unanswered: how do we find the elementary divisors 
зо that we can write out directly the Jordan canonical form? This is the content of the next section. 
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19.15.4 Computation of elementary divisors and invariant factors 


This section is based on Qiu [11] (Reading Material 3, 5 and 7). Suppose R is a ring and we can define 
a matrix over R, whose entries are elements of R. Elementary row operations of matrices over R include 
swapping two rows, multiplying a row by an invertible element of R, and adding the multiple of one row 
to another row. Elementary column operations are defined similarly. By Cramer’s rule, a matrix over R is 
invertible if and only if its determinant is an invertible element of R. 

In particular, we consider the case of R = СА], the polynomial ring over the complex number field C. 
Note the collection of invertible elements of C[A] is just C. For any square matrix A, we would like to find 
its elementary divisors. 

Qiu [11] defined the invariant factors and elementary divisors for A(A) = AI — A. The invariant factors 
and elementary divisors thus defined are the same as those defined in Gong [3]. However, proofs that these 
differently defined concepts are the same cannot be found in both sources. I don't find a nice exposition 
elsewhere of the equivalence at this moment, so ГЇЇ only focus on the algorithmic aspect which allows us to 
calculate things explicitly. 


Theorem 5. Let A be a given square matriz over the complez field С. Define А(А) = M — А. Through 
the elementary row and column manipulation of А(А) as an element of the ring C[A], А(А) can be reduced 
to a diagonal matriz. Each entry on the diagonal line can be factorized into the form of (А — X) (A — 
A2)P? -+ (A — Ак)Р*, and each (А А)? is an elementary divisor of А(А), corresponding to the рі x pi Jordan 
block 


MN 1 0 -- 0 0 
0 A 1 + 0 0 
0 0 QT 
0 0 g0" 0E 


19.15.5 Examples 


2 3 2 
Example 1. A= | 1 8 2 | . Its \-matrix can be reduced to 


-2 -14 -3 
1 0 0 
0 (4-3? Og |. 
0 m0 А 1 


So the elementary divisors are (А — 3)? and (A — 1), and the minimal polynomial is (А — 3)?(\ — 1). The 
Jordan canonical form of A is 

100 

оз 1j, 

00 3 


which can be verified by the jordan function of Matlab or the JordanDecomposition function of Mathe- 
matica. The invariant factors can be obtained by looking up the following array (see Theorem 2 or Qiu [11], 
Reading Material 7) 
A-3)? 
(А-1 


So а (А) = di) = 1 and (А) = (A= 3 — 1). 


11 
11 
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3-40 2 
4 -5 -2 4 А 
Example 2. A= |) o 3 _9|- Its matrix can be reduced to 

0 0 2 -1 

10 0 0 

01 0 0 

0 0 A+? 0 

00 0 Q-1 


So the elementary divisors are (А + 1)? and (A — 1)?, and the minimal polynomial is (А + 1)?(A — 1)?. The 
Jordan canonical form of A is 


110 0 
01.0 0 
00-1 1 
00 0 -1 


The invariant factors are di(A) = Ф(А) = da(A) = 1 and d4(A) = (A + 1? (A — 1), obtained by looking up 
the following array 

(+2)? 111 

(4-1? 111 


3-10 0 
1 1 0 0 E 

Example 3. A — з 0 5 -3V Its A-matrix can be reduced to 
4-1 3 -1 


So the elementary divisors are (A — 2)? and (A — 2), and the minimal polynomial is (A — 2)?. The Jordan 
canonical form of A is 


оо у 
оо ы 
еу о 
о бо о 


"The invariant factors аге di (A) = d2(A) = 1, 43(А) = da(A) = (A — 2)?, obtained by looking up the following 
array 
(А-2)? (4-2)? 1 1 


19.16 Numerical Range 
1. 
Proof. By Theorem 8 of Chapter 8, we can find an orthonormal basis consisting of eigenvectors of А. Let 


а, ++, à, be the eigenvalues of A (multiplicity counted) with v1, ---, v, the corresponding eigenvectors. 
For any x € X, we can find 0;(z), ---, On (£) € C such that x = Y77 6; (z)v;. Then 


[(Ат,т)| = [У 76:85) (Avi v;)| = | EE) aivi v) = (У ail: (e)? < max lail = r(A). 
ig ij 1 cs 
Combined with (2), we conclude r(A) = w(A). п 
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p» 


Proof. By definition | A] = sup),)—1 [Az]. Using the notation in the solution to Exercise 1, we have 
Ат = V 6;(x) Av; = Y ^6: (z)aiv;- 


So [Ат] = /У с la; PIA < r(A) = w(A), where the last equality comes from Exercise 1. This implies 
|A] € w(A). By Theorem 13 (ii) of Chapter Т, w(A) < [A]. Combined, we conclude w(A) = [А]. a 
3. 

Proof. To verify (7), we note 


IIa-»2-]Il r — 2) I» = Ie«- 2:6 
k k k k 
Since (Fk)” = 1, 71, ---, Fn are a permutation of гі, ---, Tn. So 


Tl - 2) = cc - ». 
к 


t= Пе =z) -etH = (1)! Пе - a). 


Combined, we conclude (1 — 2") = J], (1 — rez). To verify (8), we use (7) to get 


i-3d 
Уа-а) = ? э 


j kA 


j Tazz В a rational function over the complex plane C, which can be assumed to have the form 215} 


with P(z) and Q(z) being polynomials without common factors. Since тү, +++, т, are singularity of degree 
1 for Уут Im we conclude Q(z) = [T,(1 — rez) = 1 — 2", up to the difference of a constant factor. Since 


Xj pu zz has no zeros on complex plane, we conclude P(z) must be a constant. Combined, we conclude 


for some constant C. By letting z — 0, we can see C — n. This finishes the verification of (8). o 
4. 


Proof. If x = |; (Аз, т) = 22 + 22 + 2129. If £? + т? = 1, we have (Ат, т) = 1 + zi - sign(a2),/1 — 22. 


Calculus shows f(£) = &\/1 — 22 (-1 < € < 1) achieves maximum at & = X2. So w(A) = 1+1 = 3. 
Similarly, plain calculation shows w(A?) = 2. ü 
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